{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "Tce3stUlHN0L"
},
"source": [
"##### Copyright 2020 The TensorFlow Authors."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"cellView": "form",
"execution": {
"iopub.execute_input": "2024-04-20T11:17:18.633969Z",
"iopub.status.busy": "2024-04-20T11:17:18.633731Z",
"iopub.status.idle": "2024-04-20T11:17:18.637470Z",
"shell.execute_reply": "2024-04-20T11:17:18.636923Z"
},
"id": "tuOe1ymfHZPu"
},
"outputs": [],
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8yo62ffS5TF5"
},
"source": [
"# Using text and neural network features\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zrCwCCxhiAL7"
},
"source": [
"Welcome to the **Intermediate Colab** for **TensorFlow Decision Forests** (**TF-DF**).\n",
"In this colab, you will learn about some more advanced capabilities of **TF-DF**, including how to deal with natural language features.\n",
"\n",
"This colab assumes you are familiar with the concepts presented the [Beginner colab](beginner_colab.ipynb), notably about the installation about TF-DF.\n",
"\n",
"In this colab, you will:\n",
"\n",
"1. Train a Random Forest that consumes text features natively as categorical sets.\n",
"\n",
"1. Train a Random Forest that consumes text features using a [TensorFlow Hub](https://www.tensorflow.org/hub) module. In this setting (transfer learning), the module is already pre-trained on a large text corpus.\n",
"\n",
"1. Train a Gradient Boosted Decision Trees (GBDT) and a Neural Network together. The GBDT will consume the output of the Neural Network."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Rzskapxq7gdo"
},
"source": [
"## Setup"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:17:18.640607Z",
"iopub.status.busy": "2024-04-20T11:17:18.640390Z",
"iopub.status.idle": "2024-04-20T11:17:21.458610Z",
"shell.execute_reply": "2024-04-20T11:17:21.457811Z"
},
"id": "mZiInVYfffAb"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Collecting tensorflow_decision_forests\r\n",
" Using cached tensorflow_decision_forests-1.9.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.0 kB)\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: numpy in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow_decision_forests) (1.26.4)\r\n",
"Requirement already satisfied: pandas in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow_decision_forests) (2.2.2)\r\n",
"Requirement already satisfied: tensorflow~=2.16.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow_decision_forests) (2.16.1)\r\n",
"Requirement already satisfied: six in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow_decision_forests) (1.16.0)\r\n",
"Requirement already satisfied: absl-py in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow_decision_forests) (1.4.0)\r\n",
"Requirement already satisfied: wheel in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow_decision_forests) (0.41.2)\r\n",
"Collecting wurlitzer (from tensorflow_decision_forests)\r\n",
" Using cached wurlitzer-3.0.3-py3-none-any.whl.metadata (1.9 kB)\r\n",
"Requirement already satisfied: tf-keras~=2.16 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow_decision_forests) (2.16.0)\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: astunparse>=1.6.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (1.6.3)\r\n",
"Requirement already satisfied: flatbuffers>=23.5.26 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (24.3.25)\r\n",
"Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (0.5.4)\r\n",
"Requirement already satisfied: google-pasta>=0.1.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (0.2.0)\r\n",
"Requirement already satisfied: h5py>=3.10.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (3.11.0)\r\n",
"Requirement already satisfied: libclang>=13.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (18.1.1)\r\n",
"Requirement already satisfied: ml-dtypes~=0.3.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (0.3.2)\r\n",
"Requirement already satisfied: opt-einsum>=2.3.2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (3.3.0)\r\n",
"Requirement already satisfied: packaging in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (24.0)\r\n",
"Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (3.20.3)\r\n",
"Requirement already satisfied: requests<3,>=2.21.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (2.31.0)\r\n",
"Requirement already satisfied: setuptools in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (69.5.1)\r\n",
"Requirement already satisfied: termcolor>=1.1.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (2.4.0)\r\n",
"Requirement already satisfied: typing-extensions>=3.6.6 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (4.11.0)\r\n",
"Requirement already satisfied: wrapt>=1.11.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (1.16.0)\r\n",
"Requirement already satisfied: grpcio<2.0,>=1.24.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (1.63.0rc2)\r\n",
"Requirement already satisfied: tensorboard<2.17,>=2.16 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (2.16.2)\r\n",
"Requirement already satisfied: keras>=3.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (3.2.1)\r\n",
"Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow~=2.16.1->tensorflow_decision_forests) (0.36.0)\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: python-dateutil>=2.8.2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from pandas->tensorflow_decision_forests) (2.9.0.post0)\r\n",
"Requirement already satisfied: pytz>=2020.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from pandas->tensorflow_decision_forests) (2024.1)\r\n",
"Requirement already satisfied: tzdata>=2022.7 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from pandas->tensorflow_decision_forests) (2024.1)\r\n",
"Requirement already satisfied: rich in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from keras>=3.0.0->tensorflow~=2.16.1->tensorflow_decision_forests) (13.7.1)\r\n",
"Requirement already satisfied: namex in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from keras>=3.0.0->tensorflow~=2.16.1->tensorflow_decision_forests) (0.0.8)\r\n",
"Requirement already satisfied: optree in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from keras>=3.0.0->tensorflow~=2.16.1->tensorflow_decision_forests) (0.11.0)\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: charset-normalizer<4,>=2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorflow~=2.16.1->tensorflow_decision_forests) (3.3.2)\r\n",
"Requirement already satisfied: idna<4,>=2.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorflow~=2.16.1->tensorflow_decision_forests) (3.7)\r\n",
"Requirement already satisfied: urllib3<3,>=1.21.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorflow~=2.16.1->tensorflow_decision_forests) (2.2.1)\r\n",
"Requirement already satisfied: certifi>=2017.4.17 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorflow~=2.16.1->tensorflow_decision_forests) (2024.2.2)\r\n",
"Requirement already satisfied: markdown>=2.6.8 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.17,>=2.16->tensorflow~=2.16.1->tensorflow_decision_forests) (3.6)\r\n",
"Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.17,>=2.16->tensorflow~=2.16.1->tensorflow_decision_forests) (0.7.2)\r\n",
"Requirement already satisfied: werkzeug>=1.0.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.17,>=2.16->tensorflow~=2.16.1->tensorflow_decision_forests) (3.0.2)\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: importlib-metadata>=4.4 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from markdown>=2.6.8->tensorboard<2.17,>=2.16->tensorflow~=2.16.1->tensorflow_decision_forests) (7.1.0)\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: MarkupSafe>=2.1.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from werkzeug>=1.0.1->tensorboard<2.17,>=2.16->tensorflow~=2.16.1->tensorflow_decision_forests) (2.1.5)\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: markdown-it-py>=2.2.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from rich->keras>=3.0.0->tensorflow~=2.16.1->tensorflow_decision_forests) (3.0.0)\r\n",
"Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from rich->keras>=3.0.0->tensorflow~=2.16.1->tensorflow_decision_forests) (2.17.2)\r\n",
"Requirement already satisfied: zipp>=0.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard<2.17,>=2.16->tensorflow~=2.16.1->tensorflow_decision_forests) (3.18.1)\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: mdurl~=0.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from markdown-it-py>=2.2.0->rich->keras>=3.0.0->tensorflow~=2.16.1->tensorflow_decision_forests) (0.1.2)\r\n",
"Using cached tensorflow_decision_forests-1.9.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (15.5 MB)\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Using cached wurlitzer-3.0.3-py3-none-any.whl (7.3 kB)\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Installing collected packages: wurlitzer, tensorflow_decision_forests\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Successfully installed tensorflow_decision_forests-1.9.0 wurlitzer-3.0.3\r\n"
]
}
],
"source": [
"# Install TensorFlow Dececision Forests\n",
"!pip install tensorflow_decision_forests\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2EFndCFdoJM5"
},
"source": [
"[Wurlitzer](https://pypi.org/project/wurlitzer/) is needed to display the detailed training logs in Colabs (when using `verbose=2` in the model constructor)."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:17:21.463115Z",
"iopub.status.busy": "2024-04-20T11:17:21.462778Z",
"iopub.status.idle": "2024-04-20T11:17:23.415566Z",
"shell.execute_reply": "2024-04-20T11:17:23.414736Z"
},
"id": "L06XWRdSoLj5"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: wurlitzer in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (3.0.3)\r\n"
]
}
],
"source": [
"!pip install wurlitzer"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "i7PlfbnxYcPf"
},
"source": [
"Import the necessary libraries."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:17:23.419981Z",
"iopub.status.busy": "2024-04-20T11:17:23.419703Z",
"iopub.status.idle": "2024-04-20T11:17:25.835024Z",
"shell.execute_reply": "2024-04-20T11:17:25.834311Z"
},
"id": "RsCV2oAS7gC_"
},
"outputs": [],
"source": [
"import os\n",
"# Keep using Keras 2\n",
"os.environ['TF_USE_LEGACY_KERAS'] = '1'\n",
"\n",
"import tensorflow_decision_forests as tfdf\n",
"\n",
"import numpy as np\n",
"import pandas as pd\n",
"import tensorflow as tf\n",
"import tf_keras\n",
"import math"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "w2fsI0y5x5i5"
},
"source": [
"The hidden code cell limits the output height in colab."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"cellView": "form",
"execution": {
"iopub.execute_input": "2024-04-20T11:17:25.839123Z",
"iopub.status.busy": "2024-04-20T11:17:25.838781Z",
"iopub.status.idle": "2024-04-20T11:17:25.843204Z",
"shell.execute_reply": "2024-04-20T11:17:25.842432Z"
},
"id": "jZXB4o6Tlu0i"
},
"outputs": [],
"source": [
"#@title\n",
"\n",
"from IPython.core.magic import register_line_magic\n",
"from IPython.display import Javascript\n",
"from IPython.display import display as ipy_display\n",
"\n",
"# Some of the model training logs can cover the full\n",
"# screen if not compressed to a smaller viewport.\n",
"# This magic allows setting a max height for a cell.\n",
"@register_line_magic\n",
"def set_cell_height(size):\n",
" ipy_display(\n",
" Javascript(\"google.colab.output.setIframeHeight(0, true, {maxHeight: \" +\n",
" str(size) + \"})\"))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "M_D4Ft4o65XT"
},
"source": [
"## Use raw text as features\n",
"\n",
"TF-DF can consume [categorical-set](https://arxiv.org/pdf/2009.09991.pdf) features natively. Categorical-sets represent text features as bags of words (or n-grams).\n",
"\n",
"For example: `\"The little blue dog\" ` → `{\"the\", \"little\", \"blue\", \"dog\"}`\n",
"\n",
"In this example, you'll will train a Random Forest on the [Stanford Sentiment Treebank](https://nlp.stanford.edu/sentiment/index.html) (SST) dataset. The objective of this dataset is to classify sentences as carrying a *positive* or *negative* sentiment. You'll will use the binary classification version of the dataset curated in [TensorFlow Datasets](https://www.tensorflow.org/datasets/catalog/glue#gluesst2).\n",
"\n",
"**Note:** Categorical-set features can be expensive to train. In this colab, we will train a small Random Forest with 20 trees."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:17:25.846511Z",
"iopub.status.busy": "2024-04-20T11:17:25.846118Z",
"iopub.status.idle": "2024-04-20T11:17:27.951356Z",
"shell.execute_reply": "2024-04-20T11:17:27.950173Z"
},
"id": "SgEiFy23j14S"
},
"outputs": [],
"source": [
"# Install the TensorFlow Datasets package\n",
"!pip install tensorflow-datasets -U --quiet"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:17:27.955701Z",
"iopub.status.busy": "2024-04-20T11:17:27.955431Z",
"iopub.status.idle": "2024-04-20T11:17:31.953387Z",
"shell.execute_reply": "2024-04-20T11:17:31.952581Z"
},
"id": "uVN-j0E4Q1T3"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'idx': 163, 'label': -1, 'sentence': b'not even the hanson brothers can save it'}\n",
"{'idx': 131, 'label': -1, 'sentence': b'strong setup and ambitious goals fade as the film descends into unsophisticated scare tactics and b-film thuggery .'}\n",
"{'idx': 1579, 'label': -1, 'sentence': b'too timid to bring a sense of closure to an ugly chapter of the twentieth century .'}\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2024-04-20 11:17:31.940774: W tensorflow/core/kernels/data/cache_dataset_ops.cc:858] The calling iterator did not fully read the dataset being cached. In order to avoid unexpected truncation of the dataset, the partially cached contents of the dataset will be discarded. This can happen if you have an input pipeline similar to `dataset.cache().take(k).repeat()`. You should use `dataset.take(k).cache().repeat()` instead.\n",
"2024-04-20 11:17:31.946131: W tensorflow/core/framework/local_rendezvous.cc:404] Local rendezvous is aborting with status: OUT_OF_RANGE: End of sequence\n"
]
}
],
"source": [
"# Load the dataset\n",
"import tensorflow_datasets as tfds\n",
"all_ds = tfds.load(\"glue/sst2\")\n",
"\n",
"# Display the first 3 examples of the test fold.\n",
"for example in all_ds[\"test\"].take(3):\n",
" print({attr_name: attr_tensor.numpy() for attr_name, attr_tensor in example.items()})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "UHiQUWE2XDYN"
},
"source": [
"The dataset is modified as follows:\n",
"\n",
"1. The raw labels are integers in `{-1, 1}`, but the learning algorithm expects positive integer labels e.g. `{0, 1}`. Therefore, the labels are transformed as follows: `new_labels = (original_labels + 1) / 2`.\n",
"1. A batch-size of 64 is applied to make reading the dataset more efficient.\n",
"1. The `sentence` attribute needs to be tokenized, i.e. `\"hello world\" -> [\"hello\", \"world\"]`.\n",
"\n",
"\n",
"**Note:** This example doesn't use the `test` split of the dataset as it does not have labels. If `test` split had labels, you could concatenate the `validation` fold into the `train` one (e.g. `all_ds[\"train\"].concatenate(all_ds[\"validation\"])`).\n",
"\n",
"**Details:** Some decision forest learning algorithms do not need a validation dataset (e.g. Random Forests) while others do (e.g. Gradient Boosted Trees in some cases). Since each learning algorithm under TF-DF can use validation data differently, TF-DF handles train/validation splits internally. As a result, when you have a training and validation sets, they can always be concatenated as input to the learning algorithm."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:17:31.957721Z",
"iopub.status.busy": "2024-04-20T11:17:31.957055Z",
"iopub.status.idle": "2024-04-20T11:17:32.052722Z",
"shell.execute_reply": "2024-04-20T11:17:32.052058Z"
},
"id": "yqYDKTKdSPYw"
},
"outputs": [],
"source": [
"def prepare_dataset(example):\n",
" label = (example[\"label\"] + 1) // 2\n",
" return {\"sentence\" : tf.strings.split(example[\"sentence\"])}, label\n",
"\n",
"train_ds = all_ds[\"train\"].batch(100).map(prepare_dataset)\n",
"test_ds = all_ds[\"validation\"].batch(100).map(prepare_dataset)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YYkIjROI9w43"
},
"source": [
"Finally, train and evaluate the model as usual. TF-DF automatically detects multi-valued categorical features as categorical-set.\n"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:17:32.056273Z",
"iopub.status.busy": "2024-04-20T11:17:32.056042Z",
"iopub.status.idle": "2024-04-20T11:18:28.384322Z",
"shell.execute_reply": "2024-04-20T11:18:28.383620Z"
},
"id": "mpxTtYo39wYZ"
},
"outputs": [
{
"data": {
"application/javascript": [
"google.colab.output.setIframeHeight(0, true, {maxHeight: 300})"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Warning: The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING:absl:The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Use /tmpfs/tmp/tmpx28adgyq as temporary training directory\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Reading training dataset...\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training tensor examples:\n",
"Features: {'sentence': tf.RaggedTensor(values=Tensor(\"data:0\", shape=(None,), dtype=string), row_splits=Tensor(\"data_1:0\", shape=(None,), dtype=int64))}\n",
"Label: Tensor(\"data_2:0\", shape=(None,), dtype=int64)\n",
"Weights: None\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Normalized tensor features:\n",
" {'sentence': SemanticTensor(semantic=, tensor=tf.RaggedTensor(values=Tensor(\"data:0\", shape=(None,), dtype=string), row_splits=Tensor(\"data_1:0\", shape=(None,), dtype=int64)))}\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training dataset read in 0:00:04.709443. Found 67349 examples.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training model...\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Standard output detected as not visible to the user e.g. running in a notebook. Creating a training log redirection. If training gets stuck, try calling tfdf.keras.set_training_logs_redirection(False).\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:17:36.8175 UTC kernel.cc:771] Start Yggdrasil model training\n",
"[INFO 24-04-20 11:17:36.8176 UTC kernel.cc:772] Collect training examples\n",
"[INFO 24-04-20 11:17:36.8176 UTC kernel.cc:785] Dataspec guide:\n",
"column_guides {\n",
" column_name_pattern: \"^__LABEL$\"\n",
" type: CATEGORICAL\n",
" categorial {\n",
" min_vocab_frequency: 0\n",
" max_vocab_count: -1\n",
" }\n",
"}\n",
"default_column_guide {\n",
" categorial {\n",
" max_vocab_count: 2000\n",
" }\n",
" discretized_numerical {\n",
" maximum_num_bins: 255\n",
" }\n",
"}\n",
"ignore_columns_without_guides: false\n",
"detect_numerical_as_discretized_numerical: false\n",
"\n",
"[INFO 24-04-20 11:17:36.8179 UTC kernel.cc:391] Number of batches: 674\n",
"[INFO 24-04-20 11:17:36.8180 UTC kernel.cc:392] Number of examples: 67349\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:17:36.8602 UTC data_spec_inference.cc:305] 12816 item(s) have been pruned (i.e. they are considered out of dictionary) for the column sentence (2000 item(s) left) because min_value_count=5 and max_number_of_unique_values=2000\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:17:36.9136 UTC kernel.cc:792] Training dataset:\n",
"Number of records: 67349\n",
"Number of columns: 2\n",
"\n",
"Number of columns by type:\n",
"\tCATEGORICAL_SET: 1 (50%)\n",
"\tCATEGORICAL: 1 (50%)\n",
"\n",
"Columns:\n",
"\n",
"CATEGORICAL_SET: 1 (50%)\n",
"\t1: \"sentence\" CATEGORICAL_SET has-dict vocab-size:2001 num-oods:10187 (15.1257%) most-frequent:\"the\" 27205 (40.3941%)\n",
"\n",
"CATEGORICAL: 1 (50%)\n",
"\t0: \"__LABEL\" CATEGORICAL integerized vocab-size:3 no-ood-item\n",
"\n",
"Terminology:\n",
"\tnas: Number of non-available (i.e. missing) values.\n",
"\tood: Out of dictionary.\n",
"\tmanually-defined: Attribute whose type is manually defined by the user, i.e., the type was not automatically inferred.\n",
"\ttokenized: The attribute value is obtained through tokenization.\n",
"\thas-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.\n",
"\tvocab-size: Number of unique values.\n",
"\n",
"[INFO 24-04-20 11:17:36.9137 UTC kernel.cc:808] Configure learner\n",
"[INFO 24-04-20 11:17:36.9139 UTC kernel.cc:822] Training config:\n",
"learner: \"RANDOM_FOREST\"\n",
"features: \"^sentence$\"\n",
"label: \"^__LABEL$\"\n",
"task: CLASSIFICATION\n",
"random_seed: 123456\n",
"metadata {\n",
" framework: \"TF Keras\"\n",
"}\n",
"pure_serving_model: false\n",
"[yggdrasil_decision_forests.model.random_forest.proto.random_forest_config] {\n",
" num_trees: 30\n",
" decision_tree {\n",
" max_depth: 16\n",
" min_examples: 5\n",
" in_split_min_examples_check: true\n",
" keep_non_leaf_label_distribution: true\n",
" num_candidate_attributes: 0\n",
" missing_value_policy: GLOBAL_IMPUTATION\n",
" allow_na_conditions: false\n",
" categorical_set_greedy_forward {\n",
" sampling: 0.1\n",
" max_num_items: -1\n",
" min_item_frequency: 1\n",
" }\n",
" growing_strategy_local {\n",
" }\n",
" categorical {\n",
" cart {\n",
" }\n",
" }\n",
" axis_aligned_split {\n",
" }\n",
" internal {\n",
" sorting_strategy: PRESORTED\n",
" }\n",
" uplift {\n",
" min_examples_in_treatment: 5\n",
" split_score: KULLBACK_LEIBLER\n",
" }\n",
" }\n",
" winner_take_all_inference: true\n",
" compute_oob_performances: true\n",
" compute_oob_variable_importances: false\n",
" num_oob_variable_importances_permutations: 1\n",
" bootstrap_training_dataset: true\n",
" bootstrap_size_ratio: 1\n",
" adapt_bootstrap_size_ratio_for_maximum_training_duration: false\n",
" sampling_with_replacement: true\n",
"}\n",
"\n",
"[INFO 24-04-20 11:17:36.9143 UTC kernel.cc:825] Deployment config:\n",
"cache_path: \"/tmpfs/tmp/tmpx28adgyq/working_cache\"\n",
"num_threads: 32\n",
"try_resume_training: true\n",
"\n",
"[INFO 24-04-20 11:17:36.9145 UTC kernel.cc:887] Train model\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:17:36.9152 UTC random_forest.cc:416] Training random forest on 67349 example(s) and 1 feature(s).\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:18:08.8767 UTC random_forest.cc:802] Training of tree 1/30 (tree index:1) done accuracy:0.7412 logloss:9.32811\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:18:19.2026 UTC random_forest.cc:802] Training of tree 6/30 (tree index:27) done accuracy:0.775555 logloss:4.88012\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:18:21.1285 UTC random_forest.cc:802] Training of tree 16/30 (tree index:25) done accuracy:0.808699 logloss:1.679\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:18:22.6848 UTC random_forest.cc:802] Training of tree 26/30 (tree index:8) done accuracy:0.818557 logloss:0.904858\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:18:24.0005 UTC random_forest.cc:802] Training of tree 30/30 (tree index:6) done accuracy:0.821274 logloss:0.854486\n",
"[INFO 24-04-20 11:18:24.0013 UTC random_forest.cc:882] Final OOB metrics: accuracy:0.821274 logloss:0.854486\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:18:24.0104 UTC kernel.cc:919] Export model in log directory: /tmpfs/tmp/tmpx28adgyq with prefix da59c2f23fdb4012\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:18:24.0388 UTC kernel.cc:937] Save model in resources\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:18:24.0420 UTC abstract_model.cc:881] Model self evaluation:\n",
"Number of predictions (without weights): 67349\n",
"Number of predictions (with weights): 67349\n",
"Task: CLASSIFICATION\n",
"Label: __LABEL\n",
"\n",
"Accuracy: 0.821274 CI95[W][0.818828 0.8237]\n",
"LogLoss: : 0.854486\n",
"ErrorRate: : 0.178726\n",
"\n",
"Default Accuracy: : 0.557826\n",
"Default LogLoss: : 0.686445\n",
"Default ErrorRate: : 0.442174\n",
"\n",
"Confusion Table:\n",
"truth\\prediction\n",
" 1 2\n",
"1 19593 10187\n",
"2 1850 35719\n",
"Total: 67349\n",
"\n",
"\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:18:24.0658 UTC kernel.cc:1233] Loading model from path /tmpfs/tmp/tmpx28adgyq/model/ with prefix da59c2f23fdb4012\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:18:24.3112 UTC decision_forest.cc:734] Model loaded with 30 root(s), 43180 node(s), and 1 input feature(s).\n",
"[INFO 24-04-20 11:18:24.3113 UTC abstract_model.cc:1344] Engine \"RandomForestGeneric\" built\n",
"[INFO 24-04-20 11:18:24.3113 UTC kernel.cc:1061] Use fast generic engine\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model trained in 0:00:47.515581\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Compiling model...\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model compiled.\n"
]
},
{
"data": {
"text/plain": [
""
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%set_cell_height 300\n",
"\n",
"# Specify the model.\n",
"model_1 = tfdf.keras.RandomForestModel(num_trees=30, verbose=2)\n",
"\n",
"# Train the model.\n",
"model_1.fit(x=train_ds)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "D9FMFGzwiHCt"
},
"source": [
"In the previous logs, note that `sentence` is a `CATEGORICAL_SET` feature.\n",
"\n",
"The model is evaluated as usual:"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:18:28.388062Z",
"iopub.status.busy": "2024-04-20T11:18:28.387416Z",
"iopub.status.idle": "2024-04-20T11:18:32.437207Z",
"shell.execute_reply": "2024-04-20T11:18:32.436535Z"
},
"id": "cpf-wHl094S1"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
"1/9 [==>...........................] - ETA: 31s - loss: 0.0000e+00 - accuracy: 0.8100"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"9/9 [==============================] - 4s 4ms/step - loss: 0.0000e+00 - accuracy: 0.7638\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"BinaryCrossentropyloss: 0.0\n",
"Accuracy: 0.7637614607810974\n"
]
}
],
"source": [
"model_1.compile(metrics=[\"accuracy\"])\n",
"evaluation = model_1.evaluate(test_ds)\n",
"\n",
"print(f\"BinaryCrossentropyloss: {evaluation[0]}\")\n",
"print(f\"Accuracy: {evaluation[1]}\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YliBX4GtjncQ"
},
"source": [
"The training logs looks are follow:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:18:32.440877Z",
"iopub.status.busy": "2024-04-20T11:18:32.440603Z",
"iopub.status.idle": "2024-04-20T11:18:33.080965Z",
"shell.execute_reply": "2024-04-20T11:18:33.080264Z"
},
"id": "OnTTtBNmjpo7"
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAGwCAYAAABB4NqyAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAABb70lEQVR4nO3deVxU5f4H8M/MAMMOsssioLiLYAiImpqSaGUu1VVLRdzS3ArzppZ6s5toi1KK2u2HS3VdsrTlVqSRaLihKBqpKKjhwo4wMMg2c35/kFMTqAzbGZjP+/Wa12WeOefwPXMn58NznvM8EkEQBBAREREZEKnYBRARERG1NAYgIiIiMjgMQERERGRwGICIiIjI4DAAERERkcFhACIiIiKDwwBEREREBsdI7AL0kVqtxu3bt2FlZQWJRCJ2OURERFQPgiCgpKQErq6ukEof3MfDAFSH27dvw8PDQ+wyiIiIqAFu3LgBd3f3B27DAFQHKysrADVvoLW1tcjVEBERUX0oFAp4eHhovscfhAGoDvcue1lbWzMAERERtTL1Gb7CQdBERERkcBiAiIiIyOAwABEREZHBYQAiIiIig8MARERERAaHAYiIiIgMDgMQERERGRwGICIiIjI4DEBERERkcBiAiIiIyOAwABEREZHBYQAiIiIig8PFUImIiKjFCIKAW0V3AQDu7cxFq4MBiIiIiJpFeZUKl3NKcDFLgYtZJbiQpcClLAUU5dV4IbgD3h7rK1ptDEBERETUKIIgIEdRURN0smvCzsUsBa7mlUIt1N7eWCbB3UpVyxf6FwxAREREVG8V1Sqk55ZqQs69x52yqjq3t7MwQff2VujuYo3u7WsePk6WMDESdxiyXgSgmJgYvPvuu8jOzoafnx82bNiAoKCg+24fHR2NzZs3IzMzEw4ODnj22WcRFRUFU1NTAEBUVBT27duHS5cuwczMDP3798fatWvRtWvXljolIiKiVi+vpAKXshWaS1gXsxRIzy1FdR3dOjKpBB0dLDQhp3t7K/Robw1HKzkkEokI1T+Y6AFoz549iIyMxJYtWxAcHIzo6GiEhYUhLS0NTk5OtbbfuXMnlixZgq1bt6J///64fPkypk6dColEgnXr1gEADh8+jLlz5yIwMBDV1dVYtmwZhg8fjgsXLsDCwqKlT5GIiEivVanUuJqn1PTmXPgj8OSXVtS5vbWpkSbo9Pjjfzs7W8LUWNbClTecRBCEOq7OtZzg4GAEBgZi48aNAAC1Wg0PDw/Mnz8fS5YsqbX9vHnzcPHiRcTHx2vaFi1ahJMnTyIxMbHO35GXlwcnJyccPnwYgwYNqvV6RUUFKir+/D9ZoVDAw8MDxcXFsLa2buwpEhER6Y07ykqtcToXsxS4klOKSpW61rYSCeBtb6Hp0enmYo3urtZwtTHVy14dhUIBGxuben1/i9oDVFlZieTkZCxdulTTJpVKERoaiuPHj9e5T//+/fHZZ58hKSkJQUFBuHr1Kr7//ntMnjz5vr+nuLgYAGBnZ1fn61FRUXjzzTcbcSZERET6RaUWcC1fqTVO52JWCbIV5XVubyk3QjcXK61LWF1drGBuIvrFomYh6lnl5+dDpVLB2dlZq93Z2RmXLl2qc5/nn38e+fn5GDhwIARBQHV1NWbPno1ly5bVub1arcbLL7+MAQMGoFevXnVus3TpUkRGRmqe3+sBIiIiag0U5VW49LdByWk5JSivqt2rAwAedmZag5J7tLeGezszSKX616vTXFpdrEtISMDq1auxadMmBAcHIz09HQsXLsRbb72F5cuX19p+7ty5SE1Nve/lMQCQy+WQy+XNWTYREVGjqdUCMgvL/jJWpwSXshW4eedundubGcvQ9Y9enR7ta/63q4sVrEyNW7hy/SNqAHJwcIBMJkNOTo5We05ODlxcXOrcZ/ny5Zg8eTJmzJgBAPD19YVSqcSsWbPw+uuvQyr987a6efPm4X//+x+OHDkCd3f35jsRIiKiJqasqMal7L/16mSXQHmf+XNcbUz/cvmq5hKWp70FZAbUq6MLUQOQiYkJAgICEB8fjzFjxgCouWQVHx+PefPm1blPWVmZVsgBAJmsZtT5vfHcgiBg/vz52L9/PxISEuDt7d18J0FERNQIgiDg5p27mjE69247/72wDHXdpmRiJEVXZ6ta43VszU1avvhWTPRLYJGRkQgPD0ffvn0RFBSE6OhoKJVKREREAACmTJkCNzc3REVFAQBGjRqFdevWoU+fPppLYMuXL8eoUaM0QWju3LnYuXMnvv76a1hZWSE7OxsAYGNjAzMzM3FOlIiICDUTCZ75vQhH0/ORdL0QF7MUKCmvrnNbJyt5rXl1vB0sYCTjWuaNJXoAGj9+PPLy8rBixQpkZ2fD398fcXFxmoHRmZmZWj0+b7zxBiQSCd544w3cunULjo6OGDVqFN5++23NNps3bwYADBkyROt3bdu2DVOnTm32cyIiIrpHrRZwMVuBo+n5SEwvQNK1glqDk41lEnRytNTMqXMv8NhbcnxqcxF9HiB9pMs8AkRERH93806ZJvAcS89HgbJS63UHSzkG+tijfycH9HKz0YulIdqCVjMPEBERUVtQVFaJ4xkFSEzPx9H0fFwvKNN63dxEhmBvOwzs7IiBPg7o4myplxMJGhIGICIiIh2VV6lw5vc7SEzPR2J6Pn69Vaw1YFkmlcDfwxYDfBww0McB/h627OHRMwxARERED6FWC7iQpdD08CRdK0RFtfY4Hh8nSwz8I/AEd7TjXDt6jgGIiIioDjcKy/DLlZrAcywjH3fKqrRed7KSY6CPAwb88XCxMRWpUmoIBiAiIiLULBJ67C/jeDILtcfxWMqN0K+jneaylo8Tx/G0ZgxARERkkMqrVDh1vVATeH67rdAax2MklaBPB1sM9HHEwM726O1uC2POv9NmMAAREZFBUKkF/Ha7WBN4Tl2/g8q/jePp6mxV08PT2R5B3vawlPNrsq3i/7NERNQmCYKA3wvKNIHnWEYBiu9qj+NxsTbFwM41l7T6d7KHkzXH8RgKBiAiImozCkorasbxXKm5Pf1WkfYq6VZyI/TrZK8ZvNzJ0YLjeAwUAxAREbVadytVSLpeWDPr8pV8XMhSaL1uLJPgkQ7tagJPZwf0drPhOloEgAGIiIhaEZVawPmbRX8sM5GPM78XoVKlPY6nm4tVzXw8nR0Q5G0HcxN+1VFt/FQQEZHeEgQB1/KVmsBzLKOg1srprjY143gG+DigfycHOFpxAVF6OAYgIiLSK3klFTiWUXNJ62h6Pm4Xl2u9bmVqhP6d7DXrannZm3McD+mMAYiIiESlrKiuGcfzx8DlS9klWq+byKQI8Gyn6eXxdbOBTMrAQ43DAERERC2qWqXGuZvFmstaZzPvoEolaG3T09Vac6dWoJcdzExkIlVLbRUDEBERNStBEJCRp0TilTwkphfg5NUClFRoj+NxszXDo5pxPPawt+Q4HmpeDEBERNTkchXlOJqRj8QrBTiano9shfY4HhszYwzwsdesq9XBjuN4qGUxABERUaOVVlTj5NU/FxK9nFOq9bqJkRSBXu0wwMcBj/o4ooerNcfxkKgYgIiISGdVKjXO3SjSBJ6zmUWoVv85jkciAXq52mh6ePp6tYOpMcfxkP5gACIioocSBAFXcks1t6afuFoAZaVKaxtPe3NN4AnpaI92FiYiVUv0cAxARERUp+zichz9o4cnMT0fuSUVWq+3MzdG/z8Cz0AfB3jYmYtUKZHuGICIiAgAoCivwsmrhZrAk56rPY5HbiRFkLed5vb0Hu2tIeU4HmqlGICIiAxUZbUaZzPvaALPuZvFUP1tHE9vtz/H8TziyXE81HYwABERGQhBEJCWU6IZx3PyWiHK/jaOx9vBAgN87DHQxwH9OtrD1pzjeKhtYgAiImrDbhfd1dypdTS9APml2uN47C1MND08/X3s4d6O43jIMDAAERG1IcV3q3DiaoHmstbVPKXW62bGMq1xPN1crDiOhwwSAxARUStWUa3Cmd+LNIHn/M0i/GUYD6QSwM/DVhN4+nSwhdyI43iIGICIiFoRtVrApewSJKbXrKuVdK0A5VVqrW06OlpoAk+/jvawMTMWqVoi/cUARESk527eKfujh6cAx9LzUaCs1HrdwVKOgX+sqzXAxwGutmYiVUrUejAAERHpmeKyKhzLyNcMXr5eUKb1urmJDMHedjXranV2RBdnSy4kSqQjBiAiIpGVV6lw5vc7msBz/lYxhL+M45FJJfD3sNXcreXvYQsTI6l4BRO1AaIHoJiYGLz77rvIzs6Gn58fNmzYgKCgoPtuHx0djc2bNyMzMxMODg549tlnERUVBVNTUwDAkSNH8O677yI5ORlZWVnYv38/xowZ00JnQ0T0cGq1gAtZCk3gSbpWiIpq7XE8Pk6WmiUmgjvawcqU43iImpKoAWjPnj2IjIzEli1bEBwcjOjoaISFhSEtLQ1OTk61tt+5cyeWLFmCrVu3on///rh8+TKmTp0KiUSCdevWAQCUSiX8/Pwwbdo0jBs3rqVPiYioTjcKy5CYno/EK/k4lpGPO2VVWq87Wck1A5cH+DjAxcZUpEqJDINEEP7a0dqygoODERgYiI0bNwIA1Go1PDw8MH/+fCxZsqTW9vPmzcPFixcRHx+vaVu0aBFOnjyJxMTEWttLJJJ69QBVVFSgouLPycEUCgU8PDxQXFwMa2vrBp4dERmyO8pKHMso0PTyZBZqj+OxMJEhpJO95rKWjxPH8RA1lkKhgI2NTb2+v0XrAaqsrERycjKWLl2qaZNKpQgNDcXx48fr3Kd///747LPPkJSUhKCgIFy9ehXff/89Jk+e3KhaoqKi8OabbzbqGERk2MqrVDh9/Q5+Sc/D0fR8/HZboTWOx0gqQZ8Of47j8fOwhbGM43iIxCJaAMrPz4dKpYKzs7NWu7OzMy5dulTnPs8//zzy8/MxcOBACIKA6upqzJ49G8uWLWtULUuXLkVkZKTm+b0eICKi+1GpBfx2u1jTw3Pq+h1U/m0cT1dnq5rA09keQd72sJSLPuySiP7Qqv5rTEhIwOrVq7Fp0yYEBwcjPT0dCxcuxFtvvYXly5c3+LhyuRxyubwJKyWitkYQBPxeUKYJPMcyClB8V3scj4u1KQZ2/mNdrU72cLLmOB4ifSVaAHJwcIBMJkNOTo5We05ODlxcXOrcZ/ny5Zg8eTJmzJgBAPD19YVSqcSsWbPw+uuvQypldzIRNZ2C0gocy6hZV+uXK/m4VXRX63UruRH6dbLXDF7u5GjBcTxErYRoAcjExAQBAQGIj4/XDFJWq9WIj4/HvHnz6tynrKysVsiRyWrWtBFxLDcRtSFnM+/gh9RsJF7Jx4UshdZrxjIJHunQribwdHZAbzcbGHEcD1GrJOolsMjISISHh6Nv374ICgpCdHQ0lEolIiIiAABTpkyBm5sboqKiAACjRo3CunXr0KdPH80lsOXLl2PUqFGaIFRaWor09HTN77h27RpSUlJgZ2eHDh06tPxJElGrsTXxGt767oLW4OVuLlaawBPkZQcLjuMhahNE/S95/PjxyMvLw4oVK5CdnQ1/f3/ExcVpBkZnZmZq9fi88cYbkEgkeOONN3Dr1i04Ojpi1KhRePvttzXbnD59Go899pjm+b3BzeHh4di+fXvLnBgRtSpqtYC1cZfw0ZGrAIARPV0w0tcF/Ts5wNGK4wOJ2iJR5wHSV7rMI0BErVtltRr//OIcvkq5DQD454iumDO4E8fyELVCrWIeICIisZVWVGP2p8lITM+HkVSCtc/0xjMB7mKXRUQtgAGIiAxSbkk5Iradwm+3FTA3kWHTC49gSNfaS/AQUdvEAEREBudqXinCtyXhRuFd2FuYYFtEIHq724pdFhG1IAYgIjIoZzPvYPqO0yhUVsLT3hyfTAuCp72F2GURUQtjACIigxF/MQdzd55BeZUavd1tsHVqIBwseZcXkSFiACIig7DnVCaW7U+FSi1gcBdHbHrhEc7pQ2TA+F8/EbVpgiBgw8/pWHfwMgDgmUfcseYZX67ETmTgGICIqM2qVqmx4pvfsPNkJgBg7mOd8Orwrpzjh4gYgIiobbpbqcL8XWfx08UcSCTAqqd7YnKIl9hlEZGeYAAiojbnjrIS03ecwpnMIpgYSfHhBH+M6NVe7LKISI8wABFRm3LzThnCtyYhI08Ja1Mj/F94IIK87cQui4j0DAMQEbUZF24rMHVbEnJLKtDexhQ7pgWhi7OV2GURkR5iACKiNuFYej5e/DQZJRXV6Opshe3TAtHexkzssohITzEAEVGr982521j0eQqqVAKCvO3w8ZS+sDEzFrssItJjDEBE1Kr93y9X8e/vLgIAnvB1wbp/+MPUWCZyVUSk7xiAiKhVUqsFRP1wER//cg0AMLW/F5Y/1QMyKef4IaKHYwAiolanslqNxV+cw9cptwEAr43ohtmDO3KCQyKqNwYgImpVSsqrMPuzZBxNL4CRVIJ3nu2NcY+4i10WEbUyDEBE1GrkKsoxddspXMhSwNxEhi2TAjCoi6PYZRFRK8QAREStQkZeKcK3JuHmnbtwsDTBtqlB8HW3EbssImqlGICISO+dybyD6dtP4U5ZFbzszbFjWhA87S3ELouIWjEGICLSaz9dyMG8XWdQXqWGn7sNYqcGwsFSLnZZRNTKMQARkd7anZSJZft/hVoAhnR1RMzzj8BCzn+2iKjx+C8JEekdQRDwQfwVRP90BQDwXIA7Vo/zhbFMKnJlRNRWMAARkV6pVqmx/OtU7Eq6AQCY95gPFg3vwjl+iKhJMQARkd64W6nC/F1n8NPFXEgkwKrRvTC5n6fYZRFRG8QARER6oVBZiek7TuFsZhHkRlJ8MKEPRvRyEbssImqjGICISHQ3CssQvi0JV/OUsDEzRmx4X/T1shO7LCJqwxiAiEhUv90uxtRtp5BXUgFXG1PsmBaEzs5WYpdFRG0cAxARieZYej5mfZqM0opqdHOxwvaIILjYmIpdFhEZAAYgIhLF1ym38Orec6hSCQj2tsN/pvSFjZmx2GURkYHQi0k1YmJi4OXlBVNTUwQHByMpKemB20dHR6Nr164wMzODh4cHXnnlFZSXlzfqmETUcj4+chULd6egSiXgSd/22DEtiOGHiFqU6AFoz549iIyMxMqVK3HmzBn4+fkhLCwMubm5dW6/c+dOLFmyBCtXrsTFixcRGxuLPXv2YNmyZQ0+JhG1DLVawL//dwFvf38RABAxwAsbJvaBqbFM5MqIyNBIBEEQxCwgODgYgYGB2LhxIwBArVbDw8MD8+fPx5IlS2ptP2/ePFy8eBHx8fGatkWLFuHkyZNITExs0DErKipQUVGhea5QKODh4YHi4mJYW1s36fkSGaqKahUW7z2Pb87dBgAsHdkNswZ15ASHRNRkFAoFbGxs6vX9LWoPUGVlJZKTkxEaGqppk0qlCA0NxfHjx+vcp3///khOTtZc0rp69Sq+//57PPHEEw0+ZlRUFGxsbDQPDw+PpjpFIgKgKK9CxLZT+ObcbRhJJVg/3g8vDu7E8ENEohF1EHR+fj5UKhWcnZ212p2dnXHp0qU693n++eeRn5+PgQMHQhAEVFdXY/bs2ZpLYA055tKlSxEZGal5fq8HiIgaL0dRjvCtSbiUXQILExk2TwrAoC6OYpdFRAZO9DFAukpISMDq1auxadMmnDlzBvv27cN3332Ht956q8HHlMvlsLa21noQUeOl55Zi3KZjuJRdAgdLOfa8GMLwQ0R6QdQeIAcHB8hkMuTk5Gi15+TkwMWl7inwly9fjsmTJ2PGjBkAAF9fXyiVSsyaNQuvv/56g45JRE0v+fc7mL7jFIrKquDtYIEdEUHoYG8udllERABE7gEyMTFBQECA1oBmtVqN+Ph4hISE1LlPWVkZpFLtsmWymjtIBEFo0DGJqGkdvJCDF/7vBIrKquDnYYsvZocw/BCRXhF9IsTIyEiEh4ejb9++CAoKQnR0NJRKJSIiIgAAU6ZMgZubG6KiogAAo0aNwrp169CnTx8EBwcjPT0dy5cvx6hRozRB6GHHJKLms/NkJt746leoBeCxro6IeeERmJuI/k8NEZEW0f9VGj9+PPLy8rBixQpkZ2fD398fcXFxmkHMmZmZWj0+b7zxBiQSCd544w3cunULjo6OGDVqFN5+++16H5OImp4gCFj/0xV8GH8FAPCPvu5YPdYXRrJWN9SQiAyA6PMA6SNd5hEgIqBapcYbX6Vi96kbAIAFQ33wyuNdeJs7EbUoXb6/Re8BIqLW7W6lCvN2nkH8pVxIJcBbY3rhhWBPscsiInogBiAiarBCZSWmbT+FlBtFkBtJ8eHEPgjrybstiUj/MQARUYPcKCxD+NYkXM1XwsbMGLHhfdHXy07ssoiI6kXn0YleXl5YtWoVMjMzm6MeImoFUm8VY9zmY7iar4SbrRm+nBPC8ENErYrOAejll1/Gvn370LFjRzz++OPYvXu31kKiRNS2JV7Jx4T/nEBeSQW6uVhh30v94eNkJXZZREQ6aVAASklJQVJSErp374758+ejffv2mDdvHs6cOdMcNRKRnvg65RYitiehtKIa/Tra4fPZIXC2NhW7LCIinTX6Nviqqips2rQJr732GqqqquDr64sFCxYgIiKi1d4Cy9vgibQJgoCPf7mK1d/XLCj8ZO/2WPcPP8iNZCJXRkT0pxa5Db6qqgr79+/Htm3bcPDgQfTr1w/Tp0/HzZs3sWzZMvz000/YuXNnQw9PRHpCrRbw7+8uYuvRawCAaQO88caT3SGVts4/cIiIgAYEoDNnzmDbtm3YtWsXpFIppkyZgvXr16Nbt26abcaOHYvAwMAmLZSIWl5FtQqLPj+H/53PAgAse6IbZj7asdX27hIR3aNzAAoMDMTjjz+OzZs3Y8yYMTA2Nq61jbe3NyZMmNAkBRKROBTlVXjxk2Qcv1oAY5kE7z7rhzF93MQui4ioSegcgK5evQpPzwfP8mphYYFt27Y1uCgiEleOohzhW5NwKbsEFiYyfDS5LwZ2dhC7LCKiJqPzXWC5ubk4efJkrfaTJ0/i9OnTTVIUEYknPbcE4zYdw6XsEjhYyrHnxRCGHyJqc3QOQHPnzsWNGzdqtd+6dQtz585tkqKISBzJvxfimc3HcavoLrwdLLD/pf7o5WYjdllERE1O50tgFy5cwCOPPFKrvU+fPrhw4UKTFEVELe/Ab9mYv+ssKqrV8PewxdapgbCzMBG7LCKiZqFzD5BcLkdOTk6t9qysLBgZcWkxotbovyd/x+zPklFRrcawbk7YOTOY4YeI2jSdA9Dw4cOxdOlSFBcXa9qKioqwbNkyPP74401aHBE1L0EQsO5AGl7fnwq1AIzv64GPJgfA3IR/zBBR26bzv3LvvfceBg0aBE9PT/Tp0wcAkJKSAmdnZ3z66adNXiARNY9qlRrL9v+Kz0/fBAAsGNYZr4R25hw/RGQQdA5Abm5uOH/+PP773//i3LlzMDMzQ0REBCZOnFjnnEBEpH/KKqsxb+dZ/HwpF1IJ8O8xvng+uIPYZRERtZgG9XNbWFhg1qxZTV0LEbWAgtIKTNtxGuduFEFuJMXG5x/B4z2cxS6LiKhFNfhC/4ULF5CZmYnKykqt9qeffrrRRRFR88gsKEP4tiRcy1fC1twYseF9EeBpJ3ZZREQtrkEzQY8dOxa//vorJBIJ7i0mf2/cgEqlatoKiahJpN4qxtRtp5BfWgE3WzPsmBYEHydLscsiIhKFzneBLVy4EN7e3sjNzYW5uTl+++03HDlyBH379kVCQkIzlEhEjfXLlTyM/+g48ksr0M3FCvte6s/wQ0QGTeceoOPHj+Pnn3+Gg4MDpFIppFIpBg4ciKioKCxYsABnz55tjjqJqIH2n72JxXvPo1otIKSjPT6aEgBrU96wQESGTeceIJVKBSsrKwCAg4MDbt++DQDw9PREWlpa01ZHRA0mCAI+OpyBV/acQ7VawCg/V2yfFsjwQ0SEBvQA9erVC+fOnYO3tzeCg4PxzjvvwMTEBP/5z3/QsWPH5qiRiHSkVgt467sL2Hb0OgBgxkBvLHuiO6RSzvFDRAQ0IAC98cYbUCqVAIBVq1bhqaeewqOPPgp7e3vs2bOnyQskIt2UV6mw6PNz+O7XLADA6090x8xB/OOEiOivJMK927gaobCwEO3atWszM8gqFArY2NiguLgY1tbWYpdDVG/Fd6sw65PTOHmtEMYyCd57zg+j/d3ELouIqEXo8v2t0xigqqoqGBkZITU1Vavdzs6uzYQfotYqu7gc4z86jpPXCmEpN8L2iCCGHyKi+9DpEpixsTE6dOjAuX6I9MyVnBKEb03C7eJyOFrJsT0iED1dbcQui4hIb+l8F9jrr7+OZcuWobCwsDnqISIdnbpeiGe3HMft4nJ0dLTAvjn9GX6IiB5C50HQGzduRHp6OlxdXeHp6QkLCwut18+cOdNkxRHRg8WlZmPh7rOoqFajTwdbxIYHws7CROyyiIj0ns4BaMyYMU1eRExMDN59911kZ2fDz88PGzZsQFBQUJ3bDhkyBIcPH67V/sQTT+C7774DAOTk5OC1117DgQMHUFRUhEGDBmHDhg3o3Llzk9dOJJZPT/yOlV+nQi0Aod2dsGHiIzAzkYldFhFRq6BzAFq5cmWTFrBnzx5ERkZiy5YtCA4ORnR0NMLCwpCWlgYnJ6da2+/bt09rAdaCggL4+fnhueeeA1Az+duYMWNgbGyMr7/+GtbW1li3bh1CQ0Nx4cKFWj1WRK2NIAh4/8BlbDyUDgCYGOSBt0b3gpFM5yvaREQGq0lug2+M4OBgBAYGYuPGjQAAtVoNDw8PzJ8/H0uWLHno/tHR0VixYgWysrJgYWGBy5cvo2vXrkhNTUXPnj01x3RxccHq1asxY8aMWseoqKhARUWF5rlCoYCHhwdvgye9U6VSY9m+X7E3+SYA4OXQzlg4rDPvwiQiQjPeBg8AUqkUMpnsvg9dVFZWIjk5GaGhoVrHDw0NxfHjx+t1jNjYWEyYMEHTs3MvyJiammodUy6XIzExsc5jREVFwcbGRvPw8PDQ6TyIWkJZZTVmfnIae5NvQioBosb54uXQLgw/REQNoPMlsP3792s9r6qqwtmzZ7Fjxw68+eabOh0rPz8fKpUKzs7OWu3Ozs64dOnSQ/dPSkpCamoqYmNjNW3dunVDhw4dsHTpUnz00UewsLDA+vXrcfPmTWRlZdV5nKVLlyIyMlLz/F4PEJG+KCitwLTtp3DuZjFMjaXYOPERhPZwfviORERUJ50D0OjRo2u1Pfvss+jZsyf27NmD6dOnN0lh9REbGwtfX1+tAdPGxsbYt28fpk+fDjs7O8hkMoSGhmLkyJG439U+uVwOuVzeUmUT6SSzoAxTtp7E9YIytDM3xv+FByLAs53YZRERtWpNNmqyX79+iI+P12kfBwcHyGQy5OTkaLXn5OTAxcXlgfsqlUrs3r27zsAVEBCAlJQUFBUVISsrC3FxcSgoKOBirdTq/HqzGOM2H8X1gjK42Zrhizn9GX6IiJpAkwSgu3fv4sMPP4Sbm27T7puYmCAgIEArOKnVasTHxyMkJOSB++7duxcVFRWYNGnSfbexsbGBo6Mjrly5gtOnT9fZe0Wkrw5fzsP4/xxHfmklure3xv6X+qOTo6XYZRERtQk6XwL7+6KngiCgpKQE5ubm+Oyzz3QuIDIyEuHh4ejbty+CgoIQHR0NpVKJiIgIAMCUKVPg5uaGqKgorf1iY2MxZswY2Nvb1zrm3r174ejoiA4dOuDXX3/FwoULMWbMGAwfPlzn+ojEsO/MTfzzi/OoVgsY4GOPLZMCYGVqLHZZRERths4BaP369VoBSCqVwtHREcHBwWjXTveu+fHjxyMvLw8rVqxAdnY2/P39ERcXpxkYnZmZCalUu6MqLS0NiYmJOHDgQJ3HzMrKQmRkJHJyctC+fXtMmTIFy5cv17k2opYmCAK2HL6KtXE1NwE87eeK957zg4kR5/ghImpKos8DpI90mUeAqKmo1ALe+t8FbD92HQAw81FvLB3ZHVIpb3MnIqoPXb6/de4B2rZtGywtLTUzL9+zd+9elJWVITw8XNdDEhm88ioVIj9Pwfe/ZgMA3niyO2Y8ykH7RETNRed+9aioKDg4ONRqd3JywurVq5ukKCJDUny3ClO2JuH7X7NhLJPgw4l9GH6IiJqZzj1AmZmZ8Pb2rtXu6emJzMzMJimKyFBkFd/F1K2nkJZTAku5Ef4zOQD9fWr/gUFERE1L5x4gJycnnD9/vlb7uXPn6rwji4jqdjmnBOM2HUNaTgmcrOT4/MUQhh8iohaicw/QxIkTsWDBAlhZWWHQoEEAgMOHD2PhwoWYMGFCkxdI1BYlXSvEjB2noCivRkdHC+yICIKHnbnYZRERGQydA9Bbb72F69evY9iwYTAyqtldrVZjypQpHANEVA9xqVlYsDsFldVqPNLBFrHhgWhnYSJ2WUREBqXBt8FfuXIFKSkpMDMzg6+vLzw9PZu6NtHwNnhqLp8cv46V3/wGQQBCuztjw8Q+MDORiV0WEVGb0Ky3wd/TuXNndO7cuaG7ExkUQRDw3oE0xBzKAABMDOqAt0b3hJGMExwSEYlB5399n3nmGaxdu7ZW+zvvvFNrbiAiAqpUaiz+4rwm/EQ+3gWrx/Zi+CEiEpHO/wIfOXIETzzxRK32kSNH4siRI01SFFFboayoxowdp/FF8k3IpBKsGeeLBcM6ay0nQ0RELU/nS2ClpaUwMak9YNPY2BgKhaJJiiJqC/JLKzBt+ymcv1kMU2MpYp5/BMO6O4tdFhERoQE9QL6+vtizZ0+t9t27d6NHjx5NUhRRa/d7gRLPbD6G8zeL0c7cGDtn9mP4ISLSIzr3AC1fvhzjxo1DRkYGhg4dCgCIj4/Hrl27sHfv3iYvkKi1OX+zCBHbTqFAWQn3dmbYMS0InRwtxS6LiIj+QucANGrUKHz11VdYvXo1vvjiC5iZmaF379746aefMHjw4OaokajVSEjLxUv/PYOyShV6tLfG9ohAOFmbil0WERH9TYPnAWrLOA8QNcQXyTex5MvzqFYLGOjjgM2THoGVqbHYZRERGYwWmQeIiGoIgoBNCRl498c0AMBof1e8+6wfTIx4mzsRkb7SOQCpVCqsX78en3/+OTIzM1FZWan1emFhYZMVR6TvVGoBb377Gz45/jsAYNagjlgyohukUt7mTkSkz3T+E/XNN9/EunXrMH78eBQXFyMyMhLjxo2DVCrFv/71r2YokUg/lVepMG/nGU34Wf5UDyx7ojvDDxFRK6DzGKBOnTrhww8/xJNPPgkrKyukpKRo2k6cOIGdO3c2V60thmOA6GGKy6ow85PTSLpeCBOZFO//ww+j/FzFLouIyKDp8v2tcw9QdnY2fH19AQCWlpYoLi4GADz11FP47rvvGlAuUetyu+gunvvoGJKuF8JKboTt0wIZfoiIWhmdA5C7uzuysrIA1PQGHThwAABw6tQpyOXypq2OSM+kZZdg3KZjuJxTCicrOT6fHYL+nRzELouIiHSkcwAaO3Ys4uPjAQDz58/H8uXL0blzZ0yZMgXTpk1r8gKJ9MXJqwV4bssxZCvK0cnRAvte6o/u7XmJlIioNWr0PEAnTpzAsWPH0LlzZ4waNaqp6hIVxwDR333/axZe3pOCymo1AjzbITa8L2zNa6+JR0RE4mnReYD69euHfv36NfYwRHprx7Hr+Ne3v0EQgOE9nPHhxD4wNZaJXRYRETUCJ0Ikug9BEPDOj2nYnJABAHghuANWje4FGW9zJyJq9RiAiOpQpVLjtS/PY9+ZWwCARY93wbyhPpBIGH6IiNoCBiCiv1FWVGPOf8/gyOU8yKQSrB7bC+MDO4hdFhERNSEGIKK/yCupwLTtp/DrrWKYGcsQ80IfDO3mLHZZRETUxBiAiP5wPV+JKVuTkFlYBjsLE2ydGgh/D1uxyyIiomagcwBq165dneMgJBIJTE1N4ePjg6lTpyIiIqJJCiRqCeduFGHa9lMoUFbCw84MOyKC0NHRUuyyiIiomeg8EeKKFSsglUrx5JNP4s0338Sbb76JJ598ElKpFHPnzkWXLl0wZ84cfPzxx/U+ZkxMDLy8vGBqaorg4GAkJSXdd9shQ4ZAIpHUejz55JOabUpLSzFv3jy4u7vDzMwMPXr0wJYtW3Q9VTIQh9JyMeE/J1CgrERPV2t8Oac/ww8RURuncw9QYmIi/v3vf2P27Nla7R999BEOHDiAL7/8Er1798aHH36ImTNnPvR4e/bsQWRkJLZs2YLg4GBER0cjLCwMaWlpcHJyqrX9vn37UFlZqXleUFAAPz8/PPfcc5q2yMhI/Pzzz/jss8/g5eWFAwcO4KWXXoKrqyuefvppXU+Z2rC9p29gyb5foVILeLSzAzZPCoClnFeGiYjaOp1ngra0tERKSgp8fHy02tPT0+Hv74/S0lJkZGSgd+/eUCqVDz1ecHAwAgMDsXHjRgCAWq2Gh4cH5s+fjyVLljx0/+joaKxYsQJZWVmwsLAAAPTq1Qvjx4/H8uXLNdsFBARg5MiR+Pe///3QY3Im6LZPEATEHErHewcuAwDG9nHD2md6w8RI505RIiLSE826GrydnR2+/fbbWu3ffvst7OzsAABKpRJWVlYPPVZlZSWSk5MRGhr6Z0FSKUJDQ3H8+PF61RMbG4sJEyZowg8A9O/fH9988w1u3boFQRBw6NAhXL58GcOHD6/zGBUVFVAoFFoPartUagErvv5NE35eHNwR7z/nx/BDRGRAdO7rX758OebMmYNDhw4hKCgIQM1K8N9//71mnM3BgwcxePDghx4rPz8fKpUKzs7atxk7Ozvj0qVLD90/KSkJqampiI2N1WrfsGEDZs2aBXd3dxgZGUEqleLjjz/GoEGD6jxOVFQU3nzzzYf+Pmr9yqtUeHl3CuJ+y4ZEAqx4qgciBniLXRYREbUwnQPQzJkz0aNHD2zcuBH79u0DAHTt2hWHDx9G//79AQCLFi1q2irvIzY2Fr6+vpogds+GDRtw4sQJfPPNN/D09MSRI0cwd+5cuLq6avU23bN06VJERkZqnisUCnh4eDR7/dSyisuqMOOTUzh1/Q5MZFKsG++Hp3q7il0WERGJoEGjPQcMGIABAwY0+pc7ODhAJpMhJydHqz0nJwcuLi4P3FepVGL37t1YtWqVVvvdu3exbNky7N+/X3NnWO/evZGSkoL33nuvzgAkl8shl8sbeTakz24V3cXUrUm4klsKK7kR/jOlL0I62YtdFhERiaRRgx7Ky8sbNXbGxMQEAQEBiI+P17Sp1WrEx8cjJCTkgfvu3bsXFRUVmDRpklZ7VVUVqqqqIJVqn5pMJoNardapPmobLmUr8MymY7iSWwpnazn2zglh+CEiMnA69wCVlZXhn//8Jz7//HMUFBTUel2lUul0vMjISISHh6Nv374ICgpCdHQ0lEqlZiLFKVOmwM3NDVFRUVr7xcbGYsyYMbC31/4is7a2xuDBg7F48WKYmZnB09MThw8fxieffIJ169bpeLbU2p24WoCZn5xGSXk1fJwssWNaENxszcQui4iIRKZzAFq8eDEOHTqEzZs3Y/LkyYiJicGtW7fw0UcfYc2aNToXMH78eOTl5WHFihXIzs6Gv78/4uLiNAOjMzMza/XmpKWlITExEQcOHKjzmLt378bSpUvxwgsvoLCwEJ6ennj77bdrzV1Ebdt357Pwyp4UVKrU6OvZDv8X3he25iZil0VERHpA53mAOnTogE8++QRDhgyBtbU1zpw5Ax8fH3z66afYtWsXvv/+++aqtcVwHqDWb/vRa3jzfxcgCMDwHs74cGIfmBrLxC6LiIiaUbPOA1RYWIiOHTsCqLncVFhYCAAYOHAgjhw50oByiZqOWi0g6oeL+Ne3NeFnUr8O2DwpgOGHiIi06ByAOnbsiGvXrgEAunXrhs8//xxAzUSItra2TVockS4qq9VYtPccPjp8FQDw6vAueGt0L8iktRfvJSIiw6bzGKCIiAicO3cOgwcPxpIlSzBq1Chs3LgRVVVVHGRMoimtqMacz5Lxy5V8yKQSRI3zxT/6ci4nIiKqm85jgP7u+vXrmnFAvXv3bqq6RMUxQK1LXkkFIrYnIfWWAmbGMmx64RE81q32QrpERNS26fL93ehlr728vODl5dXYwxA1yLV8JaZsPYkbhXdhZ2GCrVMD4e9hK3ZZRESk5xo0EWJ8fDyeeuopdOrUCZ06dcJTTz2Fn376qalrI3qglBtFeGbzMdwovIsOdub4ck5/hh8iIqoXnQPQpk2bMGLECFhZWWHhwoVYuHAhrK2t8cQTTyAmJqY5aiSq5dClXEz8zwkUKivRy80aX87pD28HC7HLIiKiVkLnMUDu7u5YsmQJ5s2bp9UeExOD1atX49atW01aoBg4Bki/fX76Bpbu+xUqtYBHOztg86QAWMobfTWXiIhauWadB6ioqAgjRoyo1T58+HAUFxfrejiiehMEARvir+CfX5yHSi1gXB83xIYHMvwQEZHOdA5ATz/9NPbv31+r/euvv8ZTTz3VJEUR/Z1KLeCNr1Lx/sHLAIDZgzvh/X/4wcSoUev5EhGRgarXn84ffvih5ucePXrg7bffRkJCgmbF9hMnTuDo0aNYtGhR81RJBq28SoUFu87iwIUcSCTAyqd6YOoAb7HLIiKiVqxeY4C8vev3ZSORSHD16tVGFyU2jgHSL8v2/4qdJzNhIpMieoI/nvBtL3ZJRESkh5p8HqB7S18QtbT03FLsTsoEAPxnSgCGdOUEh0RE1HiNGkBx9OhRVFRUNFUtRLW8++MlqAXg8R7ODD9ERNRkGhWARo4c2SZueyf9lPx7IX78LQdSCfDPsK5il0NERG1IowJQI5cRI7ovQRCw5odLAIDnAjzQ2dlK5IqIiKgt4T3EpJfiL+bi1PU7kBtJ8fLjncUuh4iI2ph6BSA7Ozvk5+cDAKZNm4aSkhIAwEcffQRnZ+fmq44MkkotYG1cTe9PxABvtLcxE7kiIiJqa+oVgCorK6FQKAAAO3bsQHl5OQDg+eefh4UF11+ipvXlmZu4klsKGzNjzBncSexyiIioDarXbfAhISEYM2YMAgICIAgCFixYADOzuv8q37p1a5MWSIalvEqF9X/M9jz3sU6wMTcWuSIiImqL6hWAPvvsM6xfvx4ZGRmQSCQoLi7W9AIRNaUdx64jq7gcrjammBLiJXY5RETURtUrADk7O2PNmjUAamaF/vTTT2Fvb9+shZHhKS6rQsyhdADAK493gamxTOSKiIiordJ5GW3OCk3NZdPhdCjKq9HF2RLjHnEXuxwiImrDGnQb/OHDhzFq1Cj4+PjAx8cHTz/9NH755Zemro0MyO2iu9h29DoA4LUR3SCTSsQtiIiI2jSdA9Bnn32G0NBQmJubY8GCBZoB0cOGDcPOnTubo0YyANE/XUZltRpB3nYY2o1LXhARUfOq12rwf9W9e3fMmjULr7zyilb7unXr8PHHH+PixYtNWqAYuBp8y7qcU4IR0UegFoB9L/XHIx3aiV0SERG1Qrp8f+vcA3T16lWMGjWqVvvTTz/N8UHUIO/EpUEtACN6ujD8EBFRi9A5AHl4eCA+Pr5W+08//QQPD48mKYoMx6nrhfjpYg5kUgkWj+CCp0RE1DJ0vgts0aJFWLBgAVJSUtC/f38AwNGjR7F9+3Z88MEHTV4gtV1/XfD0H3090MnRUuSKiIjIUOgcgObMmQMXFxe8//77+PzzzwHUjAvas2cPRo8e3eQFUtt18EIOkn+/A1NjKV4O5YKnRETUcnQOQAAwduxYjB07tqlrIQNSrVLjnR/TAADTB3rD2dpU5IqIiMiQNGgeoHt27doFpVLZ6CJiYmLg5eUFU1NTBAcHIykp6b7bDhkyBBKJpNbjySef1GxT1+sSiQTvvvtuo2ulpvFF8k2k55bC1twYL3LBUyIiamGNCkAvvvgicnJyGlXAnj17EBkZiZUrV+LMmTPw8/NDWFgYcnNz69x+3759yMrK0jxSU1Mhk8nw3HPPabb56+tZWVnYunUrJBIJnnnmmUbVSk3jbqUK63+qWfB03mM+sDblgqdERNSyGhWAdJxCqE7r1q3DzJkzERERgR49emDLli0wNze/76rydnZ2cHFx0TwOHjwIc3NzrQD019ddXFzw9ddf47HHHkPHjh0bXS813rZj15CjqICbrRkmh3iKXQ4RERmgRgWgxqqsrERycjJCQ0M1bVKpFKGhoTh+/Hi9jhEbG4sJEybAwsKiztdzcnLw3XffYfr06fc9RkVFBRQKhdaDmscdZSU2J2QAABYN7wK5ERc8JSKilteoAPTDDz/Azc2twfvn5+dDpVLB2dlZq93Z2RnZ2dkP3T8pKQmpqamYMWPGfbfZsWMHrKysMG7cuPtuExUVBRsbG82D8xk1n00J6Sgpr0Y3FyuM9m/4Z4eIiKgxdA5AQ4cORVFREQBg4MCBkMvlAGqmnx46dGiTFvcwsbGx8PX1RVBQ0H232bp1K1544QWYmt7/LqOlS5eiuLhY87hx40ZzlGvwbt4pw45jvwMAXhvJBU+JiEg8Ot8Gn5CQgMrKylrt5eXlOq8I7+DgAJlMVmsgdU5ODlxcXB64r1KpxO7du7Fq1ar7bvPLL78gLS0Ne/bseeCx5HK5JshR81l/8AoqVWr062iHIV0cxS6HiIgMWL0D0Pnz5zU/X7hwQesSlUqlQlxcnM6Xw0xMTBAQEID4+HiMGTMGAKBWqxEfH4958+Y9cN+9e/eioqICkyZNuu82sbGxCAgIgJ+fn051UdO7lK3AvrM3AQBLRnaHRMLeHyIiEk+9A5C/v79mPp26LnWZmZlhw4YNOhcQGRmJ8PBw9O3bF0FBQYiOjoZSqURERAQAYMqUKXBzc0NUVJTWfrGxsRgzZgzs7e3rPK5CocDevXvx/vvv61wTNb134tIgCMATvi7w97AVuxwiIjJw9Q5A165dgyAI6NixI5KSkuDo+OclDBMTEzg5OUEm0/2OnvHjxyMvLw8rVqxAdnY2/P39ERcXpxkYnZmZCalUe6hSWloaEhMTceDAgfsed/fu3RAEARMnTtS5JmpaJ64W4OdLuTULnoZ1E7scIiIiSISmmMynjVEoFLCxsUFxcTGsra3FLqdVEwQBYzcdQ8qNIkzq1wH/HuMrdklERNRG6fL9rfMg6E8++eSBr0+ZMkXXQ1Ib9uNv2Ui5UQQzYxkWDOOCp0REpB90DkALFy7Uel5VVYWysjKYmJjA3NycAYg0qlVqvBNXs+DpzEe94WTFBU+JiEg/6DwP0J07d7QepaWlSEtLw8CBA7Fr167mqJFaqc9P38TVfCXsLEwwcxCXISEiIv3RJEthdO7cGWvWrKnVO0SGq6yyGtF/LHg6f6gPrLjgKRER6ZEmWwvMyMgIt2/fbqrDUSu37eh15JZUwMPODM8HdxC7HCIiIi06jwH65ptvtJ4LgoCsrCxs3LgRAwYMaLLCqPUqVFZiyx8Lnr46vCsXPCUiIr2jcwC6N2PzPRKJBI6Ojhg6dCgnHSQAwMaf01FSUY0e7a0xqrer2OUQERHVonMAUqvVzVEHtRE3Csvw6YnrAIAlI7tBygVPiYhIDzV4DFB+fj7y8/ObshZqA9YdvIwqlYABPvZ4tLOD2OUQERHVSacAVFRUhLlz58LBwQHOzs5wdnaGg4MD5s2bh6KiomYqkVqL324X46uUWwCA10Z044KnRESkt+p9CaywsBAhISG4desWXnjhBXTv3h1Azcrw27dvR3x8PI4dO4Z27do1W7Gk3+4tePpU7/bo7W4rdjlERET3Ve8AtGrVKpiYmCAjI0OzUOlfXxs+fDhWrVqF9evXN3mRpP+Opefj8OU8GEkleHV4V7HLISIieqB6XwL76quv8N5779UKPwDg4uKCd955B/v372/S4qh1EAQBa+IuAQCeD+4ALwcLkSsiIiJ6sHoHoKysLPTs2fO+r/fq1QvZ2dlNUhS1Lt//mo3zN4thbiLD/KFc8JSIiPRfvQOQg4MDrl+/ft/Xr127Bjs7u6aoiVqRKpUa7/5Y0/sz89GOcLSSi1wRERHRw9U7AIWFheH1119HZWVlrdcqKiqwfPlyjBgxokmLI/23+9QNXC8og4MlFzwlIqLWQ6dB0H379kXnzp0xd+5cdOvWDYIg4OLFi9i0aRMqKirw6aefNmetpGeUFdX44KcrAIAFwzrDUq7zvJpERESiqPc3lru7O44fP46XXnoJS5cuhSAIAGqWwnj88cexceNGeHh4NFuhpH9iE68hv7QCnvbmmBDIBU+JiKj10OlPdm9vb/zwww+4c+cOrlyp+cvfx8eHY38MUEFpBT46/OeCpyZGDZ5UnIiIqMU16JpFu3btEBQU1NS1UCuy4ed0KCtV8HWzwZO+7cUuh4iISCf8s510lllQhv+e/B0AFzwlIqLWiQGIdPb+wTRUqQQ82tkBA3y44CkREbU+DECkk9Rbxfg65TaAmgVPiYiIWiMGINLJ2j+WvBjt74pebjYiV0NERNQwDEBUb79cycMvV/JhLJNg0eNc8JSIiFovBiCqF7Va0PT+vBDsiQ725iJXRERE1HAMQFQv//s1C6m3FLCUG2H+UB+xyyEiImoUBiB6qMpqNd77MQ0AMGtQR9hbcsFTIiJq3RiA6KF2JWUis7AMDpZyTB/oLXY5REREjcYARA9UWlGND+Nrlj1ZGNoZFlzwlIiI2gAGIHqgj49cRYGyEt4OFpgQyMVuiYiobdCLABQTEwMvLy+YmpoiODgYSUlJ9912yJAhkEgktR5PPvmk1nYXL17E008/DRsbG1hYWCAwMBCZmZnNfSptSl5JBT7+5SoAYHFYVxjL9OLjQkRE1Giif6Pt2bMHkZGRWLlyJc6cOQM/Pz+EhYUhNze3zu337duHrKwszSM1NRUymQzPPfecZpuMjAwMHDgQ3bp1Q0JCAs6fP4/ly5fD1NS0pU6rTdjw8xWUVarg52GLkb1cxC6HiIioyUgEQRDELCA4OBiBgYHYuHEjAECtVsPDwwPz58/HkiVLHrp/dHQ0VqxYgaysLFhYWAAAJkyYAGNjY3z66acNqkmhUMDGxgbFxcWwtrZu0DFau+v5SoSuO4xqtYBdM/shpJO92CURERE9kC7f36L2AFVWViI5ORmhoaGaNqlUitDQUBw/frxex4iNjcWECRM04UetVuO7775Dly5dEBYWBicnJwQHB+Orr7667zEqKiqgUCi0HobuvQNpqFYLGNLVkeGHiIjaHFEDUH5+PlQqFZydnbXanZ2dkZ2d/dD9k5KSkJqaihkzZmjacnNzUVpaijVr1mDEiBE4cOAAxo4di3HjxuHw4cN1HicqKgo2Njaah4eHYQ/2PX+zCP87nwWJBPhnGBc8JSKitkf0MUCNERsbC19fXwQFBWna1Go1AGD06NF45ZVX4O/vjyVLluCpp57Cli1b6jzO0qVLUVxcrHncuHGjRerXR4IgYM0PNUtejPV3Qw9Xw7wESEREbZuoAcjBwQEymQw5OTla7Tk5OXBxefCgW6VSid27d2P69Om1jmlkZIQePXpotXfv3v2+d4HJ5XJYW1trPQzVL1fycSyjACYyKV55vIvY5RARETULUQOQiYkJAgICEB8fr2lTq9WIj49HSEjIA/fdu3cvKioqMGnSpFrHDAwMRFpamlb75cuX4enp2XTFt0Fq9Z+9P5NDPOFhxwVPiYiobRJ9Wt/IyEiEh4ejb9++CAoKQnR0NJRKJSIiIgAAU6ZMgZubG6KiorT2i42NxZgxY2BvX3uA7uLFizF+/HgMGjQIjz32GOLi4vDtt98iISGhJU6p1frm3G1cyFLASm6EuY9xwVMiImq7RA9A48ePR15eHlasWIHs7Gz4+/sjLi5OMzA6MzMTUql2R1VaWhoSExNx4MCBOo85duxYbNmyBVFRUViwYAG6du2KL7/8EgMHDmz282mtKqpVeO9ATa/Z7CGdYGdhInJFREREzUf0eYD0kSHOA7Q18RpW/e8CnKzkSFg8BOYmomdjIiIinbSaeYBIPyjKq7Dh55oFT18O7cLwQ0REbR4DEOHjI1dxp6wKHR0s8I++7mKXQ0RE1OwYgAxcrqIc//fLNQDAP0d0hREXPCUiIgPAbzsD90H8FdytUsHfwxZhPbngKRERGQYGIAN2Na8Uu0/VzHq9dGQ3SCQSkSsiIiJqGQxABuy9A2lQqQUM6+aE4I5c8JSIiAwHA5CBOpt5B9//ml2z4OkILnhKRESGhQHIAP11wdNnHnFHVxcrkSsiIiJqWQxABijhch5OXiuEiREXPCUiIsPEAGRgVGoBa//o/Zna3wtutmYiV0RERNTyGIAMzNcpt3ApuwRWpkZ4aUgnscshIiISBQOQASmvUuH9A5cBAC8N8YGtORc8JSIiw8QAZEA+O/E7bhXdhYu1KSIGeIldDhERkWgYgAxE8d0qbDyUDgB45fHOMDWWiVwRERGReBiADMRHhzNQVFYFHydLPPMIFzwlIiLDxgBkALKLy7H16B8LnoZxwVMiIiJ+ExqAD+Ivo7xKjQDPdni8h7PY5RAREYmOAaiNS88txZ4/FjxdwgVPiYiIADAAtXnv/ngJagEI7e6MQC87scshIiLSCwxAbVjy73fw4285kEqAf47oKnY5REREeoMBqI0ShD+XvHg2wB1dnLngKRER0T0MQG3Uz5dykXS9EHIueEpERFQLA1AbpFILWBtX0/sTMcAb7W244CkREdFfMQC1QfvO3MTlnFLYmBljzmAueEpERPR3DEBtTHmVCusO1ix4OvexTrAxNxa5IiIiIv3DANTGfHL8OrKKy+FqY4opIV5il0NERKSXGIDakOKyKsQcygAAvPJ4Fy54SkREdB8MQG3I5sMZKL5bhS7OlhjHBU+JiIjuiwGojcgqvottfyx4+tqIbpBJueQFERHR/TAAtRHrD15GRbUaQV52GNrNSexyiIiI9BoDUBtwOacEXyTfBAC8xgVPiYiIHkovAlBMTAy8vLxgamqK4OBgJCUl3XfbIUOGQCKR1Ho8+eSTmm2mTp1a6/URI0a0xKmI4p24NKgFIKynMwI824ldDhERkd4zEruAPXv2IDIyElu2bEFwcDCio6MRFhaGtLQ0ODnVvpSzb98+VFZWap4XFBTAz88Pzz33nNZ2I0aMwLZt2zTP5XJ5852EiE5dL8RPF2sWPF0c1k3scoiIiFoF0XuA1q1bh5kzZyIiIgI9evTAli1bYG5ujq1bt9a5vZ2dHVxcXDSPgwcPwtzcvFYAksvlWtu1a9f2ekYEQcCaPxY8HR/oAR8nS5ErIiIiah1EDUCVlZVITk5GaGiopk0qlSI0NBTHjx+v1zFiY2MxYcIEWFhYaLUnJCTAyckJXbt2xZw5c1BQUHDfY1RUVEChUGg9WoODF3KQ/PsdmBpLsXAYFzwlIiKqL1EDUH5+PlQqFZydnbXanZ2dkZ2d/dD9k5KSkJqaihkzZmi1jxgxAp988gni4+Oxdu1aHD58GCNHjoRKparzOFFRUbCxsdE8PDw8Gn5SLaRapcY7P6YBAKYN8IaLjanIFREREbUeoo8BaozY2Fj4+voiKChIq33ChAman319fdG7d2906tQJCQkJGDZsWK3jLF26FJGRkZrnCoVC70PQl2duIj23FLbmxniRC54SERHpRNQeIAcHB8hkMuTk5Gi15+TkwMXF5YH7KpVK7N69G9OnT3/o7+nYsSMcHByQnp5e5+tyuRzW1tZaD312t1KF9QevAADmPeYDGzMueEpERKQLUQOQiYkJAgICEB8fr2lTq9WIj49HSEjIA/fdu3cvKioqMGnSpIf+nps3b6KgoADt27dvdM36YPux68hWlMPN1gyTQzzFLoeIiKjVEf0usMjISHz88cfYsWMHLl68iDlz5kCpVCIiIgIAMGXKFCxdurTWfrGxsRgzZgzs7e212ktLS7F48WKcOHEC169fR3x8PEaPHg0fHx+EhYW1yDk1p6KySmxKqOnJWjS8C+RGXPCUiIhIV6KPARo/fjzy8vKwYsUKZGdnw9/fH3FxcZqB0ZmZmZBKtXNaWloaEhMTceDAgVrHk8lkOH/+PHbs2IGioiK4urpi+PDheOutt9rEXECbEjJQUl6Nbi5WGO3vJnY5RERErZJEEARB7CL0jUKhgI2NDYqLi/VqPNCtort47L0EVFarsS0iEI915ZpfRERE9+jy/S36JTCqv/UHL6OyWo1+He0wpIuj2OUQERG1WgxArcSlbAW+PFOz4OmSkd254CkREVEjMAC1Eu/GpUEQgCd8XeDvYSt2OURERK0aA1ArcPJqAeIv5UImleDV4V3FLoeIiKjVYwDSc4IgYE1czYKnEwI90NGRC54SERE1FgOQnvvxt2yczSyCmbEMC4d1FrscIiKiNoEBSI9Vq9R4J65mwdMZj3rDyZoLnhIRETUFBiA99vnpm7iar0Q7c2PMGtRR7HKIiIjaDAYgPVVWWY3ony4DAOYP7QwrUy54SkRE1FQYgPTUtqPXkVtSAfd2ZnihXwexyyEiImpTGID0UKGyElsSMgAArw7vygVPiYiImhgDkB6KOZSOkopqdG9vjaf9XMUuh4iIqM1hANIzNwrL8Onx3wEAS0Z2g1TKJS+IiIiaGgOQnll/8DIqVWoM8LHHoM4OYpdDRETUJjEA6ZELtxXYn3ILAPDaiG5c8JSIiKiZMADpkXd+vARBAJ7q3R693W3FLoeIiKjNYgDSE8cy8pGQlgcjLnhKRETU7BiA9IAgCFj7Q82Cp88Hd4CXg4XIFREREbVtDEB64IfUbJy7WQxzExnmD+WCp0RERM2NAUhkVSo13v2xZsHTmY92hKOVXOSKiIiI2j4GIJHtOXUD1/KVsLcwwUwueEpERNQiGIBEpKyoRvRPVwAAC4Z1hqXcSOSKiIiIDAMDkIhiE68hv7QCHezMMTGIC54SERG1FAYgkRSUVuCjw38seBrWFSZG/L+CiIiopfBbVyQbfk6HslKFXm7WeMq3vdjlEBERGRQGIBFkFpThvyf/WPB0RHcueEpERNTCGIBE8P7BNFSpBDza2QEDueApERFRi2MAamGpt4rxdcptADULnhIREVHLYwBqYWvjapa8eNrPFb3cbESuhoiIyDAxALWgxCv5+OVKPoxlXPCUiIhITJx5rwUVKCtgbWqEcY+4o4O9udjlEBERGSy96AGKiYmBl5cXTE1NERwcjKSkpPtuO2TIEEgkklqPJ598ss7tZ8+eDYlEgujo6Gaqvv5G+7vhl38OxSuhXcQuhYiIyKCJHoD27NmDyMhIrFy5EmfOnIGfnx/CwsKQm5tb5/b79u1DVlaW5pGamgqZTIbnnnuu1rb79+/HiRMn4Orq2tynUW825sawMTcWuwwiIiKDJnoAWrduHWbOnImIiAj06NEDW7Zsgbm5ObZu3Vrn9nZ2dnBxcdE8Dh48CHNz81oB6NatW5g/fz7++9//wtiYgYOIiIj+JGoAqqysRHJyMkJDQzVtUqkUoaGhOH78eL2OERsbiwkTJsDCwkLTplarMXnyZCxevBg9e/Z86DEqKiqgUCi0HkRERNR2iRqA8vPzoVKp4OzsrNXu7OyM7Ozsh+6flJSE1NRUzJgxQ6t97dq1MDIywoIFC+pVR1RUFGxsbDQPDw+P+p8EERERtTqiXwJrjNjYWPj6+iIoKEjTlpycjA8++ADbt2+HRFK/JSaWLl2K4uJizePGjRvNVTIRERHpAVEDkIODA2QyGXJycrTac3Jy4OLi8sB9lUoldu/ejenTp2u1//LLL8jNzUWHDh1gZGQEIyMj/P7771i0aBG8vLzqPJZcLoe1tbXWg4iIiNouUQOQiYkJAgICEB8fr2lTq9WIj49HSEjIA/fdu3cvKioqMGnSJK32yZMn4/z580hJSdE8XF1dsXjxYvz444/Nch5ERETUuog+EWJkZCTCw8PRt29fBAUFITo6GkqlEhEREQCAKVOmwM3NDVFRUVr7xcbGYsyYMbC3t9dqt7e3r9VmbGwMFxcXdO3K2ZeJiIhIDwLQ+PHjkZeXhxUrViA7Oxv+/v6Ii4vTDIzOzMyEVKrdUZWWlobExEQcOHBAjJKJiIiolZMIgiCIXYS+USgUsLGxQXFxMccDERERtRK6fH+36rvAiIiIiBqCAYiIiIgMDgMQERERGRwGICIiIjI4ot8Fpo/ujQvnmmBEREStx73v7frc38UAVIeSkhIA4JpgRERErVBJSQlsbGweuA1vg6+DWq3G7du3YWVlVWs9MYVCAQ8PD9y4cYO3yOuA71vD8H3THd+zhuH71jB83xqmud43QRBQUlICV1fXWnMI/h17gOoglUrh7u7+wG24ZljD8H1rGL5vuuN71jB83xqG71vDNMf79rCen3s4CJqIiIgMDgMQERERGRwGIB3J5XKsXLkScrlc7FJaFb5vDcP3TXd8zxqG71vD8H1rGH143zgImoiIiAwOe4CIiIjI4DAAERERkcFhACIiIiKDwwBEREREBocBSEcxMTHw8vKCqakpgoODkZSUJHZJeu1f//oXJBKJ1qNbt25il6VXjhw5glGjRsHV1RUSiQRfffWV1uuCIGDFihVo3749zMzMEBoaiitXrohTrB552Ps2derUWp+9ESNGiFOsnoiKikJgYCCsrKzg5OSEMWPGIC0tTWub8vJyzJ07F/b29rC0tMQzzzyDnJwckSrWD/V534YMGVLr8zZ79myRKtYPmzdvRu/evTWTHYaEhOCHH37QvC72Z40BSAd79uxBZGQkVq5ciTNnzsDPzw9hYWHIzc0VuzS91rNnT2RlZWkeiYmJYpekV5RKJfz8/BATE1Pn6++88w4+/PBDbNmyBSdPnoSFhQXCwsJQXl7ewpXql4e9bwAwYsQIrc/erl27WrBC/XP48GHMnTsXJ06cwMGDB1FVVYXhw4dDqVRqtnnllVfw7bffYu/evTh8+DBu376NcePGiVi1+OrzvgHAzJkztT5v77zzjkgV6wd3d3esWbMGycnJOH36NIYOHYrRo0fjt99+A6AHnzWB6i0oKEiYO3eu5rlKpRJcXV2FqKgoEavSbytXrhT8/PzELqPVACDs379f81ytVgsuLi7Cu+++q2krKioS5HK5sGvXLhEq1E9/f98EQRDCw8OF0aNHi1JPa5GbmysAEA4fPiwIQs1ny9jYWNi7d69mm4sXLwoAhOPHj4tVpt75+/smCIIwePBgYeHCheIV1Uq0a9dO+L//+z+9+KyxB6ieKisrkZycjNDQUE2bVCpFaGgojh8/LmJl+u/KlStwdXVFx44d8cILLyAzM1PsklqNa9euITs7W+tzZ2Njg+DgYH7u6iEhIQFOTk7o2rUr5syZg4KCArFL0ivFxcUAADs7OwBAcnIyqqqqtD5v3bp1Q4cOHfh5+4u/v2/3/Pe//4WDgwN69eqFpUuXoqysTIzy9JJKpcLu3buhVCoREhKiF581LoZaT/n5+VCpVHB2dtZqd3Z2xqVLl0SqSv8FBwdj+/bt6Nq1K7KysvDmm2/i0UcfRWpqKqysrMQuT+9lZ2cDQJ2fu3uvUd1GjBiBcePGwdvbGxkZGVi2bBlGjhyJ48ePQyaTiV2e6NRqNV5++WUMGDAAvXr1AlDzeTMxMYGtra3Wtvy8/amu9w0Ann/+eXh6esLV1RXnz5/Ha6+9hrS0NOzbt0/EasX366+/IiQkBOXl5bC0tMT+/fvRo0cPpKSkiP5ZYwCiZjVy5EjNz71790ZwcDA8PT3x+eefY/r06SJWRm3dhAkTND/7+vqid+/e6NSpExISEjBs2DARK9MPc+fORWpqKsfk6eh+79usWbM0P/v6+qJ9+/YYNmwYMjIy0KlTp5YuU2907doVKSkpKC4uxhdffIHw8HAcPnxY7LIAcBB0vTk4OEAmk9UaoZ6TkwMXFxeRqmp9bG1t0aVLF6Snp4tdSqtw77PFz13jdezYEQ4ODvzsAZg3bx7+97//4dChQ3B3d9e0u7i4oLKyEkVFRVrb8/NW437vW12Cg4MBwOA/byYmJvDx8UFAQACioqLg5+eHDz74QC8+awxA9WRiYoKAgADEx8dr2tRqNeLj4xESEiJiZa1LaWkpMjIy0L59e7FLaRW8vb3h4uKi9blTKBQ4efIkP3c6unnzJgoKCgz6sycIAubNm4f9+/fj559/hre3t9brAQEBMDY21vq8paWlITMz06A/bw973+qSkpICAAb9eauLWq1GRUWFfnzWWmSodRuxe/duQS6XC9u3bxcuXLggzJo1S7C1tRWys7PFLk1vLVq0SEhISBCuXbsmHD16VAgNDRUcHByE3NxcsUvTGyUlJcLZs2eFs2fPCgCEdevWCWfPnhV+//13QRAEYc2aNYKtra3w9ddfC+fPnxdGjx4teHt7C3fv3hW5cnE96H0rKSkRXn31VeH48ePCtWvXhJ9++kl45JFHhM6dOwvl5eVily6aOXPmCDY2NkJCQoKQlZWleZSVlWm2mT17ttChQwfh559/Fk6fPi2EhIQIISEhIlYtvoe9b+np6cKqVauE06dPC9euXRO+/vproWPHjsKgQYNErlxcS5YsEQ4fPixcu3ZNOH/+vLBkyRJBIpEIBw4cEARB/M8aA5CONmzYIHTo0EEwMTERgoKChBMnTohdkl4bP3680L59e8HExERwc3MTxo8fL6Snp4tdll45dOiQAKDWIzw8XBCEmlvhly9fLjg7OwtyuVwYNmyYkJaWJm7ReuBB71tZWZkwfPhwwdHRUTA2NhY8PT2FmTNnGvwfK3W9XwCEbdu2aba5e/eu8NJLLwnt2rUTzM3NhbFjxwpZWVniFa0HHva+ZWZmCoMGDRLs7OwEuVwu+Pj4CIsXLxaKi4vFLVxk06ZNEzw9PQUTExPB0dFRGDZsmCb8CIL4nzWJIAhCy/Q1EREREekHjgEiIiIig8MARERERAaHAYiIiIgMDgMQERERGRwGICIiIjI4DEBERERkcBiAiIiIyOAwABEREZHBYQAiIr10/fp1SCQSzZpK+uDSpUvo168fTE1N4e/vL3Y5RNQIDEBEVKepU6dCIpFgzZo1Wu1fffUVJBKJSFWJa+XKlbCwsEBaWprWIo5/NWTIELz88sstWxgR6YwBiIjuy9TUFGvXrsWdO3fELqXJVFZWNnjfjIwMDBw4EJ6enrC3t2/wcQRBQHV1dYP3J6LGYwAiovsKDQ2Fi4sLoqKi7rvNv/71r1qXg6Kjo+Hl5aV5PnXqVIwZMwarV6+Gs7MzbG1tsWrVKlRXV2Px4sWws7ODu7s7tm3bVuv4ly5dQv/+/WFqaopevXrh8OHDWq+npqZi5MiRsLS0hLOzMyZPnoz8/HzN60OGDMG8efPw8ssvw8HBAWFhYXWeh1qtxqpVq+Du7g65XA5/f3/ExcVpXpdIJEhOTsaqVasgkUjwr3/9q9Yxpk6disOHD+ODDz6ARCKBRCLB9evXkZCQAIlEgh9++AEBAQGQy+VITEyEWq1GVFQUvL29YWZmBj8/P3zxxRc6nd8XX3wBX19fmJmZwd7eHqGhoVAqlXWeIxH9iQGIiO5LJpNh9erV2LBhA27evNmoY/3888+4ffs2jhw5gnXr1mHlypV46qmn0K5dO5w8eRKzZ8/Giy++WOv3LF68GIsWLcLZs2cREhKCUaNGoaCgAABQVFSEoUOHok+fPjh9+jTi4uKQk5ODf/zjH1rH2LFjB0xMTHD06FFs2bKlzvo++OADvP/++3jvvfdw/vx5hIWF4emnn8aVK1cAAFlZWejZsycWLVqErKwsvPrqq3UeIyQkBDNnzkRWVhaysrLg4eGheX3JkiVYs2YNLl68iN69eyMqKgqffPIJtmzZgt9++w2vvPIKJk2apAl5Dzu/rKwsTJw4EdOmTcPFixeRkJCAcePGgWtcE9VDi607T0StSnh4uDB69GhBEAShX79+wrRp0wRBEIT9+/cLf/2nY+XKlYKfn5/WvuvXrxc8PT21juXp6SmoVCpNW9euXYVHH31U87y6ulqwsLAQdu3aJQiCIFy7dk0AIKxZs0azTVVVleDu7i6sXbtWEARBeOutt4Thw4dr/e4bN24IAIS0tDRBEARh8ODBQp8+fR56vq6ursLbb7+t1RYYGCi89NJLmud+fn7CypUrH3icwYMHCwsXLtRqO3TokABA+OqrrzRt5eXlgrm5uXDs2DGtbadPny5MnDixXueXnJwsABCuX7/+0PMjIm1GYoYvImod1q5di6FDh9bZ61FfPXv2hFT6Z6ezs7MzevXqpXkuk8lgb2+P3Nxcrf1CQkI0PxsZGaFv3764ePEiAODcuXM4dOgQLC0ta/2+jIwMdOnSBQAQEBDwwNoUCgVu376NAQMGaLUPGDAA586dq+cZPlzfvn01P6enp6OsrAyPP/641jaVlZXo06cPgIef3/DhwzFs2DD4+voiLCwMw4cPx7PPPot27do1Wc1EbRUDEBE91KBBgxAWFoalS5di6tSpWq9JpdJal1yqqqpqHcPY2FjruUQiqbNNrVbXu67S0lKMGjUKa9eurfVa+/btNT9bWFjU+5jN6a91lJaWAgC+++47uLm5aW0nl8s12zzo/GQyGQ4ePIhjx47hwIED2LBhA15//XWcPHkS3t7ezXgmRK0fAxAR1cuaNWvg7++Prl27arU7OjoiOzsbgiBobo9vyrl7Tpw4gUGDBgEAqqurkZycjHnz5gEAHnnkEXz55Zfw8vKCkVHD/zmztraGq6srjh49isGDB2vajx49iqCgIJ2OZWJiApVK9dDtevToAblcjszMTK3f+Vf1OT+JRIIBAwZgwIABWLFiBTw9PbF//35ERkbqVDeRoeEgaCKqF19fX7zwwgv48MMPtdqHDBmCvLw8vPPOO8jIyEBMTAx++OGHJvu9MTEx2L9/Py5duoS5c+fizp07mDZtGgBg7ty5KCwsxMSJE3Hq1ClkZGTgxx9/RERERL1CyF8tXrwYa9euxZ49e5CWloYlS5YgJSUFCxcu1Ok4Xl5eOHnyJK5fv478/Pz79mhZWVnh1VdfxSuvvIIdO3YgIyMDZ86cwYYNG7Bjx456nd/JkyexevVqnD59GpmZmdi3bx/y8vLQvXt3nWomMkQMQERUb6tWrar1hd69e3ds2rQJMTEx8PPzQ1JSUqPGCv3dmjVrsGbNGvj5+SExMRHffPMNHBwcAEDTa6NSqTB8+HD4+vri5Zdfhq2trdZ4o/pYsGABIiMjsWjRIvj6+iIuLg7ffPMNOnfurNNxXn31VchkMvTo0QOOjo7IzMy877ZvvfUWli9fjqioKHTv3h0jRozAd999p7l89bDzs7a2xpEjR/DEE0+gS5cueOONN/D+++9j5MiROtVMZIgkwt8v3hMRERG1cewBIiIiIoPDAEREREQGhwGIiIiIDA4DEBERERkcBiAiIiIyOAxAREREZHAYgIiIiMjgMAARERGRwWEAIiIiIoPDAEREREQGhwGIiIiIDM7/A5E/H+rn3+2zAAAAAElFTkSuQmCC",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"logs = model_1.make_inspector().training_logs()\n",
"plt.plot([log.num_trees for log in logs], [log.evaluation.accuracy for log in logs])\n",
"plt.xlabel(\"Number of trees\")\n",
"plt.ylabel(\"Out-of-bag accuracy\")\n",
"pass"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "d4qJ0ig3kgic"
},
"source": [
"More trees would probably be beneficial (I am sure of it because I tried :p)."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Iil_oyOhCNx6"
},
"source": [
"## Use a pretrained text embedding\n",
"\n",
"The previous example trained a Random Forest using raw text features. This example will use a pre-trained TF-Hub embedding to convert text features into a dense embedding, and then train a Random Forest on top of it. In this situation, the Random Forest will only \"see\" the numerical output of the embedding (i.e. it will not see the raw text). \n",
"\n",
"In this experiment, will use the [Universal-Sentence-Encoder](https://tfhub.dev/google/universal-sentence-encoder/4). Different pre-trained embeddings might be suited for different types of text (e.g. different language, different task) but also for other type of structured features (e.g. images).\n",
"\n",
"**Note:** This embedding is large (1GB) and therefore the final model will be slow to run (compared to classical decision tree inference).\n",
"\n",
"The embedding module can be applied in one of two places:\n",
"\n",
"1. During the dataset preparation.\n",
"2. In the pre-processing stage of the model.\n",
"\n",
"The second option is often preferable: Packaging the embedding in the model makes the model easier to use (and harder to misuse).\n",
"\n",
"First install TF-Hub:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:18:33.084882Z",
"iopub.status.busy": "2024-04-20T11:18:33.084423Z",
"iopub.status.idle": "2024-04-20T11:18:35.276538Z",
"shell.execute_reply": "2024-04-20T11:18:35.275411Z"
},
"id": "QfYGXim_DskC"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: tensorflow-hub in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (0.16.1)\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: numpy>=1.12.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow-hub) (1.26.4)\r\n",
"Requirement already satisfied: protobuf>=3.19.6 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow-hub) (3.20.3)\r\n",
"Requirement already satisfied: tf-keras>=2.14.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow-hub) (2.16.0)\r\n",
"Requirement already satisfied: tensorflow<2.17,>=2.16 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tf-keras>=2.14.1->tensorflow-hub) (2.16.1)\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: absl-py>=1.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (1.4.0)\r\n",
"Requirement already satisfied: astunparse>=1.6.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (1.6.3)\r\n",
"Requirement already satisfied: flatbuffers>=23.5.26 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (24.3.25)\r\n",
"Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (0.5.4)\r\n",
"Requirement already satisfied: google-pasta>=0.1.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (0.2.0)\r\n",
"Requirement already satisfied: h5py>=3.10.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (3.11.0)\r\n",
"Requirement already satisfied: libclang>=13.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (18.1.1)\r\n",
"Requirement already satisfied: ml-dtypes~=0.3.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (0.3.2)\r\n",
"Requirement already satisfied: opt-einsum>=2.3.2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (3.3.0)\r\n",
"Requirement already satisfied: packaging in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (24.0)\r\n",
"Requirement already satisfied: requests<3,>=2.21.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (2.31.0)\r\n",
"Requirement already satisfied: setuptools in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (69.5.1)\r\n",
"Requirement already satisfied: six>=1.12.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (1.16.0)\r\n",
"Requirement already satisfied: termcolor>=1.1.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (2.4.0)\r\n",
"Requirement already satisfied: typing-extensions>=3.6.6 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (4.11.0)\r\n",
"Requirement already satisfied: wrapt>=1.11.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (1.16.0)\r\n",
"Requirement already satisfied: grpcio<2.0,>=1.24.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (1.63.0rc2)\r\n",
"Requirement already satisfied: tensorboard<2.17,>=2.16 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (2.16.2)\r\n",
"Requirement already satisfied: keras>=3.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (3.2.1)\r\n",
"Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (0.36.0)\r\n",
"Requirement already satisfied: wheel<1.0,>=0.23.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from astunparse>=1.6.0->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (0.41.2)\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: rich in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from keras>=3.0.0->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (13.7.1)\r\n",
"Requirement already satisfied: namex in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from keras>=3.0.0->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (0.0.8)\r\n",
"Requirement already satisfied: optree in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from keras>=3.0.0->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (0.11.0)\r\n",
"Requirement already satisfied: charset-normalizer<4,>=2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (3.3.2)\r\n",
"Requirement already satisfied: idna<4,>=2.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (3.7)\r\n",
"Requirement already satisfied: urllib3<3,>=1.21.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (2.2.1)\r\n",
"Requirement already satisfied: certifi>=2017.4.17 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (2024.2.2)\r\n",
"Requirement already satisfied: markdown>=2.6.8 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.17,>=2.16->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (3.6)\r\n",
"Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.17,>=2.16->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (0.7.2)\r\n",
"Requirement already satisfied: werkzeug>=1.0.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.17,>=2.16->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (3.0.2)\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: importlib-metadata>=4.4 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from markdown>=2.6.8->tensorboard<2.17,>=2.16->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (7.1.0)\r\n",
"Requirement already satisfied: MarkupSafe>=2.1.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from werkzeug>=1.0.1->tensorboard<2.17,>=2.16->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (2.1.5)\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: markdown-it-py>=2.2.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from rich->keras>=3.0.0->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (3.0.0)\r\n",
"Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from rich->keras>=3.0.0->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (2.17.2)\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: zipp>=0.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard<2.17,>=2.16->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (3.18.1)\r\n",
"Requirement already satisfied: mdurl~=0.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from markdown-it-py>=2.2.0->rich->keras>=3.0.0->tensorflow<2.17,>=2.16->tf-keras>=2.14.1->tensorflow-hub) (0.1.2)\r\n"
]
}
],
"source": [
"!pip install --upgrade tensorflow-hub"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "kNSEhJgjEXww"
},
"source": [
"Unlike before, you don't need to tokenize the text."
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:18:35.281111Z",
"iopub.status.busy": "2024-04-20T11:18:35.280804Z",
"iopub.status.idle": "2024-04-20T11:18:35.342789Z",
"shell.execute_reply": "2024-04-20T11:18:35.342218Z"
},
"id": "pS5SYqoScbOc"
},
"outputs": [],
"source": [
"def prepare_dataset(example):\n",
" label = (example[\"label\"] + 1) // 2\n",
" return {\"sentence\" : example[\"sentence\"]}, label\n",
"\n",
"train_ds = all_ds[\"train\"].batch(100).map(prepare_dataset)\n",
"test_ds = all_ds[\"validation\"].batch(100).map(prepare_dataset)\n"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:18:35.346035Z",
"iopub.status.busy": "2024-04-20T11:18:35.345759Z",
"iopub.status.idle": "2024-04-20T11:19:30.805044Z",
"shell.execute_reply": "2024-04-20T11:19:30.804385Z"
},
"id": "zHEsd8q_ESpC"
},
"outputs": [
{
"data": {
"application/javascript": [
"google.colab.output.setIframeHeight(0, true, {maxHeight: 300})"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Warning: The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING:absl:The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Use /tmpfs/tmp/tmpv_kuhy0b as temporary training directory\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Reading training dataset...\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training dataset read in 0:00:24.071412. Found 67349 examples.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training model...\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:19:25.9064 UTC kernel.cc:1233] Loading model from path /tmpfs/tmp/tmpv_kuhy0b/model/ with prefix 36a4a9d3f10743e4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model trained in 0:00:13.926431\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Compiling model...\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:19:27.6042 UTC decision_forest.cc:734] Model loaded with 100 root(s), 565608 node(s), and 512 input feature(s).\n",
"[INFO 24-04-20 11:19:27.6044 UTC abstract_model.cc:1344] Engine \"RandomForestOptPred\" built\n",
"[INFO 24-04-20 11:19:27.6045 UTC kernel.cc:1061] Use fast generic engine\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model compiled.\n"
]
},
{
"data": {
"text/plain": [
""
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%set_cell_height 300\n",
"\n",
"import tensorflow_hub as hub\n",
"# NNLM (https://tfhub.dev/google/nnlm-en-dim128/2) is also a good choice.\n",
"hub_url = \"https://tfhub.dev/google/universal-sentence-encoder/4\"\n",
"embedding = hub.KerasLayer(hub_url)\n",
"\n",
"sentence = tf_keras.layers.Input(shape=(), name=\"sentence\", dtype=tf.string)\n",
"embedded_sentence = embedding(sentence)\n",
"\n",
"raw_inputs = {\"sentence\": sentence}\n",
"processed_inputs = {\"embedded_sentence\": embedded_sentence}\n",
"preprocessor = tf_keras.Model(inputs=raw_inputs, outputs=processed_inputs)\n",
"\n",
"model_2 = tfdf.keras.RandomForestModel(\n",
" preprocessing=preprocessor,\n",
" num_trees=100)\n",
"\n",
"model_2.fit(x=train_ds)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:19:30.808565Z",
"iopub.status.busy": "2024-04-20T11:19:30.808272Z",
"iopub.status.idle": "2024-04-20T11:19:32.752540Z",
"shell.execute_reply": "2024-04-20T11:19:32.751811Z"
},
"id": "xPLoDqiFKY18"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
"1/9 [==>...........................] - ETA: 14s - loss: 0.0000e+00 - accuracy: 0.8000"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"4/9 [============>.................] - ETA: 0s - loss: 0.0000e+00 - accuracy: 0.8100 "
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"7/9 [======================>.......] - ETA: 0s - loss: 0.0000e+00 - accuracy: 0.7914"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"9/9 [==============================] - 2s 18ms/step - loss: 0.0000e+00 - accuracy: 0.7878\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"BinaryCrossentropyloss: 0.0\n",
"Accuracy: 0.7878440618515015\n"
]
}
],
"source": [
"model_2.compile(metrics=[\"accuracy\"])\n",
"evaluation = model_2.evaluate(test_ds)\n",
"\n",
"print(f\"BinaryCrossentropyloss: {evaluation[0]}\")\n",
"print(f\"Accuracy: {evaluation[1]}\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WPsD3LyaMLHm"
},
"source": [
"Note that categorical sets represent text differently from a dense embedding, so it may be useful to use both strategies jointly."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "37AGJamzboZQ"
},
"source": [
"## Train a decision tree and neural network together\n",
"\n",
"The previous example used a pre-trained Neural Network (NN) to \n",
"process the text features before passing them to the Random Forest. This example will train both the Neural Network and the Random Forest from scratch.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YJIxGwwzMkFl"
},
"source": [
"TF-DF's Decision Forests do not back-propagate gradients ([although this is the subject of ongoing research](https://arxiv.org/abs/2007.14761)). Therefore, the training happens in two stages:\n",
"\n",
"1. Train the neural-network as a standard classification task:\n",
"\n",
"```\n",
"example → [Normalize] → [Neural Network*] → [classification head] → prediction\n",
"*: Training.\n",
"```\n",
"\n",
"2. Replace the Neural Network's head (the last layer and the soft-max) with a Random Forest. Train the Random Forest as usual:\n",
"\n",
"```\n",
"example → [Normalize] → [Neural Network] → [Random Forest*] → prediction\n",
"*: Training.\n",
"```\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YSIvuAhzbjWO"
},
"source": [
"### Prepare the dataset\n",
"\n",
"This example uses the [Palmer's Penguins](https://allisonhorst.github.io/palmerpenguins/articles/intro.html) dataset. See the [Beginner colab](beginner_colab.ipynb) for details."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "InUot_K2b3Mz"
},
"source": [
"First, download the raw data:"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:19:32.756956Z",
"iopub.status.busy": "2024-04-20T11:19:32.756326Z",
"iopub.status.idle": "2024-04-20T11:19:33.024241Z",
"shell.execute_reply": "2024-04-20T11:19:33.023124Z"
},
"id": "rNyaeCx0b1be"
},
"outputs": [],
"source": [
"!wget -q https://storage.googleapis.com/download.tensorflow.org/data/palmer_penguins/penguins.csv -O /tmp/penguins.csv"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "pNPZzQekb9z_"
},
"source": [
"Load a dataset into a Pandas Dataframe."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:19:33.028635Z",
"iopub.status.busy": "2024-04-20T11:19:33.028092Z",
"iopub.status.idle": "2024-04-20T11:19:33.047052Z",
"shell.execute_reply": "2024-04-20T11:19:33.046432Z"
},
"id": "9lA3peQ4sa9a"
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" species \n",
" island \n",
" bill_length_mm \n",
" bill_depth_mm \n",
" flipper_length_mm \n",
" body_mass_g \n",
" sex \n",
" year \n",
" \n",
" \n",
" \n",
" \n",
" 0 \n",
" Adelie \n",
" Torgersen \n",
" 39.1 \n",
" 18.7 \n",
" 181.0 \n",
" 3750.0 \n",
" male \n",
" 2007 \n",
" \n",
" \n",
" 1 \n",
" Adelie \n",
" Torgersen \n",
" 39.5 \n",
" 17.4 \n",
" 186.0 \n",
" 3800.0 \n",
" female \n",
" 2007 \n",
" \n",
" \n",
" 2 \n",
" Adelie \n",
" Torgersen \n",
" 40.3 \n",
" 18.0 \n",
" 195.0 \n",
" 3250.0 \n",
" female \n",
" 2007 \n",
" \n",
" \n",
"
\n",
"
"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/tmpfs/tmp/__autograph_generated_filetvdtpwer.py:63: UserWarning: Input dict contained keys ['bill_depth_mm', 'flipper_length_mm', 'body_mass_g', 'sex', 'year'] which did not match any model input. They will be ignored by the model.\n",
" ag__.converted_call(ag__.ld(warnings).warn, (ag__.converted_call('Input dict contained keys {} which did not match any model input. They will be ignored by the model.'.format, ([ag__.ld(n) for n in ag__.converted_call(ag__.ld(tensors).keys, (), None, fscope) if ag__.ld(n) not in ag__.ld(ref_input_names)],), None, fscope),), dict(stacklevel=2), fscope)\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n",
"I0000 00:00:1713611983.477059 22965 service.cc:145] XLA service 0x7ff3c4155d00 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:\n",
"I0000 00:00:1713611983.477103 22965 service.cc:153] StreamExecutor device (0): Tesla T4, Compute Capability 7.5\n",
"I0000 00:00:1713611983.477109 22965 service.cc:153] StreamExecutor device (1): Tesla T4, Compute Capability 7.5\n",
"I0000 00:00:1713611983.477112 22965 service.cc:153] StreamExecutor device (2): Tesla T4, Compute Capability 7.5\n",
"I0000 00:00:1713611983.477115 22965 service.cc:153] StreamExecutor device (3): Tesla T4, Compute Capability 7.5\n",
"I0000 00:00:1713611983.604064 22965 device_compiler.h:188] Compiled cluster using XLA! This line is logged at most once for the lifetime of the process.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
"1/1 [==============================] - ETA: 0s - loss: 1.0737 - accuracy: 0.7137"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"1/1 [==============================] - 7s 7s/step - loss: 1.0737 - accuracy: 0.7137 - val_loss: 1.0569 - val_accuracy: 0.7500\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 2/10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
"1/1 [==============================] - ETA: 0s - loss: 1.0696 - accuracy: 0.7177"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"1/1 [==============================] - 0s 24ms/step - loss: 1.0696 - accuracy: 0.7177 - val_loss: 1.0529 - val_accuracy: 0.7500\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 3/10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
"1/1 [==============================] - ETA: 0s - loss: 1.0655 - accuracy: 0.7177"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"1/1 [==============================] - 0s 22ms/step - loss: 1.0655 - accuracy: 0.7177 - val_loss: 1.0489 - val_accuracy: 0.7500\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 4/10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
"1/1 [==============================] - ETA: 0s - loss: 1.0614 - accuracy: 0.7218"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"1/1 [==============================] - 0s 22ms/step - loss: 1.0614 - accuracy: 0.7218 - val_loss: 1.0450 - val_accuracy: 0.7500\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 5/10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
"1/1 [==============================] - ETA: 0s - loss: 1.0574 - accuracy: 0.7258"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"1/1 [==============================] - 0s 23ms/step - loss: 1.0574 - accuracy: 0.7258 - val_loss: 1.0410 - val_accuracy: 0.7500\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 6/10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
"1/1 [==============================] - ETA: 0s - loss: 1.0533 - accuracy: 0.7298"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"1/1 [==============================] - 0s 22ms/step - loss: 1.0533 - accuracy: 0.7298 - val_loss: 1.0371 - val_accuracy: 0.7708\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 7/10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
"1/1 [==============================] - ETA: 0s - loss: 1.0494 - accuracy: 0.7339"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"1/1 [==============================] - 0s 22ms/step - loss: 1.0494 - accuracy: 0.7339 - val_loss: 1.0332 - val_accuracy: 0.7708\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 8/10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
"1/1 [==============================] - ETA: 0s - loss: 1.0454 - accuracy: 0.7379"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"1/1 [==============================] - 0s 25ms/step - loss: 1.0454 - accuracy: 0.7379 - val_loss: 1.0293 - val_accuracy: 0.7708\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 9/10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
"1/1 [==============================] - ETA: 0s - loss: 1.0415 - accuracy: 0.7419"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"1/1 [==============================] - 0s 22ms/step - loss: 1.0415 - accuracy: 0.7419 - val_loss: 1.0254 - val_accuracy: 0.7812\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 10/10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
"1/1 [==============================] - ETA: 0s - loss: 1.0376 - accuracy: 0.7460"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"1/1 [==============================] - 0s 22ms/step - loss: 1.0376 - accuracy: 0.7460 - val_loss: 1.0217 - val_accuracy: 0.7812\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model: \"model_1\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"__________________________________________________________________________________________________\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" Layer (type) Output Shape Param # Connected to \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"==================================================================================================\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" island (InputLayer) [(None, 1)] 0 [] \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" bill_length_mm (InputLayer [(None, 1)] 0 [] \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" ) \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" string_lookup (StringLooku (None, 1) 0 ['island[0][0]'] \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" p) \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" normalization (Normalizati (None, 1) 3 ['bill_length_mm[0][0]'] \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" on) \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" category_encoding (Categor (None, 32) 0 ['string_lookup[0][0]'] \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" yEncoding) \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" concatenate (Concatenate) (None, 33) 0 ['normalization[0][0]', \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" 'category_encoding[0][0]'] \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" dense (Dense) (None, 16) 544 ['concatenate[0][0]'] \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" dense_1 (Dense) (None, 3) 51 ['dense[0][0]'] \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"==================================================================================================\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Total params: 598 (2.34 KB)\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Trainable params: 595 (2.32 KB)\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Non-trainable params: 3 (16.00 Byte)\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"__________________________________________________________________________________________________\n"
]
}
],
"source": [
"%set_cell_height 300\n",
"\n",
"nn_model.compile(\n",
" optimizer=tf_keras.optimizers.Adam(),\n",
" loss=tf_keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
" metrics=[\"accuracy\"])\n",
"\n",
"nn_model.fit(x=train_ds, validation_data=test_ds, epochs=10)\n",
"nn_model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "N2mgMZOpgMQp"
},
"source": [
"The neural network layers are shared between the two models. So now that the neural network is trained the decision forest model will be fit to the trained output of the neural network layers:"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:19:44.597280Z",
"iopub.status.busy": "2024-04-20T11:19:44.597036Z",
"iopub.status.idle": "2024-04-20T11:19:45.485510Z",
"shell.execute_reply": "2024-04-20T11:19:45.484799Z"
},
"id": "JAc9niXqud7V"
},
"outputs": [
{
"data": {
"application/javascript": [
"google.colab.output.setIframeHeight(0, true, {maxHeight: 300})"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Reading training dataset...\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training dataset read in 0:00:00.463558. Found 248 examples.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training model...\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model trained in 0:00:00.043113\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Compiling model...\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 24-04-20 11:19:45.0975 UTC kernel.cc:1233] Loading model from path /tmpfs/tmp/tmpgsdeyzk5/model/ with prefix ac3d07af419249e2\n",
"[INFO 24-04-20 11:19:45.1138 UTC decision_forest.cc:734] Model loaded with 300 root(s), 5640 node(s), and 8 input feature(s).\n",
"[INFO 24-04-20 11:19:45.1138 UTC abstract_model.cc:1344] Engine \"RandomForestGeneric\" built\n",
"[INFO 24-04-20 11:19:45.1138 UTC kernel.cc:1061] Use fast generic engine\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model compiled.\n"
]
},
{
"data": {
"text/plain": [
""
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%set_cell_height 300\n",
"\n",
"df_and_nn_model.fit(x=train_ds)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HF8Ru2HSv1a5"
},
"source": [
"Now evaluate the composed model:"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:19:45.488708Z",
"iopub.status.busy": "2024-04-20T11:19:45.488459Z",
"iopub.status.idle": "2024-04-20T11:19:45.749038Z",
"shell.execute_reply": "2024-04-20T11:19:45.748386Z"
},
"id": "EPMlcObzuw89"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
"1/1 [==============================] - ETA: 0s - loss: 0.0000e+00 - accuracy: 0.9479"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"1/1 [==============================] - 0s 240ms/step - loss: 0.0000e+00 - accuracy: 0.9479\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Evaluation: [0.0, 0.9479166865348816]\n"
]
}
],
"source": [
"df_and_nn_model.compile(metrics=[\"accuracy\"])\n",
"print(\"Evaluation:\", df_and_nn_model.evaluate(test_ds))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "awiHEznlv5sI"
},
"source": [
"Compare it to the Neural Network alone:"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"execution": {
"iopub.execute_input": "2024-04-20T11:19:45.752389Z",
"iopub.status.busy": "2024-04-20T11:19:45.752141Z",
"iopub.status.idle": "2024-04-20T11:19:45.774439Z",
"shell.execute_reply": "2024-04-20T11:19:45.773731Z"
},
"id": "--ompWYTvxM-"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
"1/1 [==============================] - ETA: 0s - loss: 1.0217 - accuracy: 0.7812"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"1/1 [==============================] - 0s 14ms/step - loss: 1.0217 - accuracy: 0.7812\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Evaluation : [1.021651029586792, 0.78125]\n"
]
}
],
"source": [
"print(\"Evaluation :\", nn_model.evaluate(test_ds))"
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "intermediate_colab.ipynb",
"provenance": [],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.19"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
![](\"https://www.tensorflow.org/images/tf_logo_32px.png\"){kind=logo}
View on TensorFlow.org\n",
" ![](\"https://www.tensorflow.org/images/colab_logo_32px.png\"){kind=logo}
Run in Google Colab\n",
" ![](\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\")
View on GitHub\n",
" ![](\"https://www.tensorflow.org/images/download_logo_32px.png\"){kind=logo}
Download notebook\n",
" ![](\"https://www.tensorflow.org/images/hub_logo_32px.png\"){kind=logo}
See TF Hub model\n",
"