Train.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I90qRc4S1vgm"
      },
      "source": [
        "### This is my implementation of a Neural Network from scratch for a homework in Sharif University of Technology (Majority of codes are from different places)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "y8GOy2dW19v7"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import os\n",
        "import matplotlib.pyplot as plt\n",
        "from model import *\n",
        "from linear import *\n",
        "from module import *\n",
        "from optimizer import *\n",
        "from relu import *\n",
        "from sgd import *\n",
        "from sigmoid import *\n",
        "from softmax_crossentropy import *"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oSbUOPo11-q1"
      },
      "source": [
        "## 1. Modules\n",
        "In this problem you see some classes for typical modules used in neural networks. These modules include:\n",
        "\n",
        "1. **`ReLU`**: ReLU activation function\n",
        "\n",
        "2. **`Sigmoid`**: Sigmoid activation function\n",
        "\n",
        "3. **`SoftmaxCrossEntropy`**: A module which represents the softmax activation function followed by a cross entropy loss function.\n",
        "\n",
        "4. **`Linear`**: Fully connected layer which multiplies the input by a weight matrix and adds a bias term to it\n",
        "\n",
        "In this problem you have to implement `forward()` and `backward()` functions of the modules in the above list. Then your implemented codes will be tested here in this notebook.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vKHSlGoN4aZi"
      },
      "source": [
        "### 1.1. Activation Functions\n",
        "Open files `relu.py` and `sigmoid.py`. There you can see **`ReLU`** and **`Sigmoid`** classes. \n",
        "Implement `forward()` and `backward()` functions of theses classes.\n",
        "\n",
        "Here are some tips that might help you:<br/>\n",
        "**1)** `forward()` function of the layers takes an input called `x` which is a numpy 2d-array with shape `(N, D)`. Implement the forward pass corresponding to considered function. Store the results of forward pass in `out`.<br/>\n",
        "**2)** `backward()` function of the layers takes a parameter called `dout`. `dout` is the gradient of loss w.r.t. the ouput of layer in the forward pass. Provided `dout`, you have to compute the gradient w.r.t. the inputs (i.e. `x`)  to the layer. Store the results in `dx`. <br/>\n",
        "**3)** For implementing backward pass, you may need some of the variables computed during forward pass; Save these variables in the `self.cache` attribute of the layer during forward pass and use them in the backward pass.<br/>\n",
        "**4)** Test your implementation with the following cells. You should see a small value as error between your funnctions outputs and `correct output`, `correct_dx` for each activation functions. We get error of orders 1e-8 and less."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "rP-EErtUyC9R"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Relu Test Cell:\n",
            "Relative error forward pass: 1.2327591637534371e-08\n",
            "Relative error backward pass: 0.0\n"
          ]
        }
      ],
      "source": [
        "###########################################################################\n",
        "#                             Relu Test                                   #\n",
        "###########################################################################\n",
        "np.random.seed(22)\n",
        "N = 5\n",
        "d = 6\n",
        "x = np.random.randn(N,d)\n",
        "relu = ReLU('test')\n",
        "print('Relu Test Cell:')\n",
        "output = relu.forward(x)\n",
        "correct_output = [[0.,         0. ,        1.08179168, 0.,         0.  ,       0.        ],\n",
        "                  [0.9188215 , 0.  ,       0.62649346, 0. ,        0.02885512, 0.,        ],\n",
        "                  [0.58775221, 0.75231758, 0.   ,      1.05597241 ,0.74775027, 1.06467659],\n",
        "                  [1.52012959, 0.  ,       1.85998989, 0.  ,       0. ,        0.337325  ],\n",
        "                  [1.04672873 ,0.62914334, 0.36305909, 0.5557497,  0.,         0.02369477]]\n",
        "\n",
        "print('Relative error forward pass:', np.linalg.norm(output - correct_output))\n",
        "    \n",
        "dx = relu.backward(np.ones((N,d), dtype=np.float32))\n",
        "correct_dx = [[0., 0. ,1. ,0. ,0., 0.],\n",
        "              [1., 0. ,1., 0. ,1., 0.],\n",
        "              [1. ,1. ,0. ,1., 1., 1.],\n",
        "              [1. ,0. ,1., 0., 0. ,1.],\n",
        "              [1. ,1. ,1. ,1. ,0., 1.]]\n",
        "print('Relative error backward pass:', np.linalg.norm(dx - correct_dx))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "8bxGWZGDanc4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Sigmoid Test Cell:\n",
            "Relative error forward pass: 1.3652868830496266e-08\n",
            "Relative error backward pass: 1.515199259019241e-08\n"
          ]
        }
      ],
      "source": [
        "###########################################################################\n",
        "#                             sigmoid Test                                #\n",
        "###########################################################################\n",
        "np.random.seed(22)\n",
        "N=5\n",
        "d=6\n",
        "x = np.random.randn(N,d)\n",
        "sigmoid = Sigmoid('test')\n",
        "print('Sigmoid Test Cell:')    \n",
        "output = sigmoid.forward(x)\n",
        "correct_output = [[0.4770287,  0.18795539, 0.74683289, 0.44045266, 0.37962761, 0.26849495],\n",
        "                  [0.71480192, 0.24905997, 0.65169394, 0.36319727, 0.50721328, 0.44256287],\n",
        "                  [0.64284923, 0.67968348, 0.25759572, 0.74192012, 0.6786883,  0.74358323],\n",
        "                  [0.82055756, 0.18413151, 0.86529577, 0.16817555, 0.34387488, 0.58354059],\n",
        "                  [0.74014623, 0.65229519, 0.58978075, 0.63546853, 0.25189151, 0.50592342]]\n",
        "    \n",
        "print('Relative error forward pass:', np.linalg.norm(output - correct_output))\n",
        "    \n",
        "dx = sigmoid.backward(np.ones((N,d), dtype=np.float32))\n",
        "correct_dx = [[0.24947232, 0.15262816, 0.18907352, 0.24645411, 0.23551049, 0.19640541],\n",
        "              [0.20386014, 0.1870291,  0.22698895, 0.23128501, 0.24994797, 0.24670098],\n",
        "              [0.2295941,  0.21771385, 0.19124017, 0.19147466, 0.21807049, 0.19066721],\n",
        "              [0.14724285, 0.1502271,  0.116559,   0.13989254, 0.22562495, 0.24302097],\n",
        "              [0.19232979, 0.22680617, 0.24193942, 0.23164828, 0.18844218, 0.24996491]]\n",
        "    \n",
        "print('Relative error backward pass:', np.linalg.norm(dx - correct_dx))\n",
        "    "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VhU008dXEGab"
      },
      "source": [
        "### 1.2. Softmax activation with Cross Entropy loss\n",
        "\n",
        "You have to implement a numerically stable version of softmax in this problem.\n",
        "\n",
        "Open file `softmax_crossentropy.py`. There you see a class for Softmax activation with Cross Entropy loss used in neural networks.\n",
        " \n",
        "\n",
        "Implement `forward()` and `backward()` function of SoftmaxCrossentropy class corresponding to forward and backward pass of softmax activation followed by cross entropy loss. Test your implementation with the following functions. The order of outputs should be 1e-8 or smaller."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "gnm4K_L-EbJh"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Softmax with Cross Entropy Test Cell:\n",
            "1.6883967462546619\n",
            "Loss relative error: 0.0\n",
            "Gradient relative error: 1.695804576567757e-08\n"
          ]
        }
      ],
      "source": [
        "###########################################################################\n",
        "#                  Softmax with Cross Entropy Test                        #\n",
        "###########################################################################\n",
        "np.random.seed(22)\n",
        "N=5\n",
        "d=6\n",
        "  \n",
        "x = np.linspace(1000, 1015, num=N*d).reshape(N,d)\n",
        "y = np.random.randint(0, d, (N, ))\n",
        "    \n",
        "softmax_ce = SoftmaxCrossentropy('test')\n",
        "print('Softmax with Cross Entropy Test Cell:')    \n",
        "loss, _ = softmax_ce.forward(x, y=y)\n",
        "dx = softmax_ce.backward()\n",
        "    \n",
        "    \n",
        "correct_loss = 1.6883967462546619\n",
        "print('Loss relative error:', np.abs(loss - correct_loss))\n",
        "    \n",
        "correct_dx = [[ 0.00636809,  0.0106818,   0.01791759,  0.03005485,  0.05041383, -0.11543615],\n",
        "              [ 0.00636809,  0.0106818,   0.01791759,  0.03005485, -0.14958617,  0.08456385],\n",
        "              [ 0.00636809,  0.0106818,   0.01791759,  0.03005485, -0.14958617,  0.08456385],\n",
        "              [-0.19363191,  0.0106818,   0.01791759,  0.03005485,  0.05041383,  0.08456385],\n",
        "              [ 0.00636809,  0.0106818,   0.01791759,  0.03005485, -0.14958617,  0.08456385]]\n",
        "print('Gradient relative error:', np.linalg.norm(dx - correct_dx))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GY2kXfpbMwPO"
      },
      "source": [
        "### 1.3. Linear\n",
        "Open file `linear.py`. There you see a class for linear (or fully connected) layer used in neural networks.<br/> \n",
        "\n",
        "Implement `forward()` and `backward()` functions corresponding to forward and backward pass of a linear layer.<br/>\n",
        " \n",
        "We have implemented `__init__()` constructor for you. You can see `W`, `b`, `dW`, `db` variables in this class.\n",
        "\n",
        "In the forward pass, You have to output $y=XW+b$ in which $W$ and $b$ are parameters of the layer. In the backward pass you get $\\large{\\frac{\\partial{loss}}{\\partial{y}}}$ as `dout`; You have to compute $\\large{\\frac{\\partial{loss}}{\\partial{X}}}$, $\\large{\\frac{\\partial{loss}}{\\partial{W}}}$ and $\\large{\\frac{\\partial{loss}}{\\partial{b}}}$ and save them in `dx`, `dW`, and `db`. The output of the `backward()` function will be `dx`. \n",
        "\n",
        "Note that a linear layer with the shape of weights $(D, K)$ represents a layer in a MLP with $K$ neurons (or units). In other words, it takes a $D$-dimensional input and gives back a $K$-dimensional ouput.\n",
        "\n",
        "Test your implementation with the following functions. The order of ouputs should be 1e-8 or smaller."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "y0FyQoJ8NiDi"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Relative error forward pass: 1.223759443671177e-08\n",
            "Relative error dx: 1.2514524575828446e-08\n",
            "Relative error dw: 7.449439033436333e-09\n",
            "Relative error db: 3.529100840673046e-09\n"
          ]
        }
      ],
      "source": [
        "###########################################################################\n",
        "#                         Linear Test                            #\n",
        "###########################################################################\n",
        "np.random.seed(22)\n",
        "D = 4\n",
        "K = 3\n",
        "N = 5\n",
        "x = np.random.randn(N,D)\n",
        "linear = Linear('test', D, K, l2_coef=1.)\n",
        "output = linear.forward(x)\n",
        "    \n",
        "correct_output = [[-0.51242952, -1.47921276, -2.32943713],\n",
        "                  [-1.17901283, -2.60908172,  0.54809823],\n",
        "                  [ 0.74600461, -2.24752841, -1.1013558 ],\n",
        "                  [ 0.75284837,  1.80111973, -2.27011589],\n",
        "                  [ 2.03171234, -3.05396933,  1.35213333]]\n",
        "    \n",
        "print('Relative error forward pass:', np.linalg.norm(output - correct_output))\n",
        "    \n",
        "dout = np.random.randn(N, K)\n",
        "dx = linear.backward(dout)\n",
        "    \n",
        "correct_dx = [[-0.25519113, -0.09724317,  0.280189,    0.87644613],\n",
        "              [ 1.20379991, -0.78816259, -1.27930227, -4.1952743 ],\n",
        "              [-0.77808532, -0.05005675, -3.14028536, -8.02818572],\n",
        "              [ 0.95446653, -1.90375857,  1.62080372,  3.57597736],\n",
        "              [ 2.86716776, -1.39892213,  0.31786772, -0.88234943]]\n",
        "print('Relative error dx:', np.linalg.norm(dx - correct_dx))\n",
        "    \n",
        "correct_dW = [[ 3.33629487, -4.43357113, -1.89100503],\n",
        "              [ 1.31103323,  2.17687036, -2.33906146],\n",
        "              [ 1.69538051, -0.89256682, -0.86018824],\n",
        "              [-0.87944724,  7.48073741, -7.0605863 ]]\n",
        "print('Relative error dw:', np.linalg.norm(linear.dW - correct_dW))\n",
        "    \n",
        "correct_db = [-1.02223284, -3.61915576, -0.16696389]\n",
        "print('Relative error db:', np.linalg.norm(linear.db - correct_db))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DbIbV_oku_fq"
      },
      "source": [
        "## 2. Optimization Algorithm\n",
        "\n",
        "Open file `sgd.py`.<br/> \n",
        "you have to implement `SGD` class.\n",
        "\n",
        "Test your implementation with the following cells. The order of errors should be 1e-8 or smaller."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "AMwdZUmqRMQO",
        "outputId": "e7454185-e7fe-4eae-d03e-310677864cce"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "W Relative error: 1.4426196588173724e-08\n",
            "b Relative error:  5.771526066321419e-09\n"
          ]
        }
      ],
      "source": [
        "###########################################################################\n",
        "#                           SGD Test                             #\n",
        "###########################################################################\n",
        "N, D = 5, 4\n",
        "np.random.seed(22)\n",
        "linear = Linear('test', N, D, l2_coef=1.)\n",
        "linear.dW = np.random.randn(N, D)\n",
        "linear.db = np.random.randn(D,)\n",
        "\n",
        "sgd = SGD(1e-2)\n",
        "\n",
        "sgd.update(linear)\n",
        "\n",
        "correct_W = [[-0.10241721, -1.46964209,  1.07816109, -0.24488267],\n",
        "             [-0.48024364, -1.00250896,  0.89382974, -1.0787318 ],\n",
        "             [ 0.62884208, -0.56053815,  0.03772041, -0.22940006],\n",
        "             [ 0.58673243,  0.75482687, -1.05771443,  1.06682406],\n",
        "             [ 0.74180098,  1.07106567,  1.53120796, -1.50966439]]\n",
        "correct_b = [ 1.86566377, -1.59381353, -0.62684131,  0.33332912]\n",
        "\n",
        "print('W Relative error:', np.linalg.norm(correct_W - linear.W))\n",
        "print('b Relative error: ', np.linalg.norm(correct_b - linear.b))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B78R9RWgKWFD"
      },
      "source": [
        "## 3. Test Network on CIFAR10\n",
        "CIFAR10 is a dataset of 50,000 32x32x3 color training images and 10,000 test images, labeled over 10 categories (labels are 0 to 9)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "bedhflJ8RE_n"
      },
      "outputs": [],
      "source": [
        "from tensorflow.keras.datasets import cifar10"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YzulAL2CTz5Y"
      },
      "source": [
        "### 3.1. Load Data\n",
        "download the dataset using cifar10.load_data() and store the results into (X_train, y_train), (X_val, y_val)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "_lx0mp8vOfZf",
        "scrolled": true
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Downloading data from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz\n",
            "170498071/170498071 [==============================] - 74s 0us/step\n",
            "(50000, 32, 32, 3) (50000, 1)\n",
            "(10000, 32, 32, 3) (10000, 1)\n"
          ]
        }
      ],
      "source": [
        "(X_train, y_train), (X_val, y_val)= cifar10.load_data()\n",
        "print(X_train.shape, y_train.shape)\n",
        "print(X_val.shape, y_val.shape)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3po6F_CYOfZf"
      },
      "source": [
        "flatten the images (reshape the images to single dimension vectors)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "cOFG4QMjOfZf"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "((50000, 3072), (10000, 3072))"
            ]
          },
          "execution_count": 12,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "X_train_flattened = X_train.reshape(X_train.shape[0], 32 * 32 * 3) # flatten each image in X_train\n",
        "X_val_flattened = X_val.reshape(X_val.shape[0], 32 * 32 * 3) # flatten each image in X_val\n",
        "X_train_flattened.shape, X_val_flattened.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Fm6WcPL8OfZg"
      },
      "source": [
        "convert the labels to scalers (using `np.squeeze()` method)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "jp-q_Uw2OfZg"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "((50000,), (10000,))"
            ]
          },
          "execution_count": 14,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "y_train_squeezed =  np.squeeze(y_train) # squeeze y_train\n",
        "y_val_squeezed = np.squeeze(y_val) # squeeze y_val\n",
        "y_train_squeezed.shape, y_val_squeezed.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z0oq5JkMOfZg"
      },
      "source": [
        "### 3.2. Normalization\n",
        " Run the cell below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "EsctzHrpOfZg"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "255 0\n",
            "255 0\n"
          ]
        }
      ],
      "source": [
        "print(X_train_flattened.max(), X_train_flattened.min())\n",
        "print(X_val_flattened.max(), X_val_flattened.min())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JeiQ7zurOfZh"
      },
      "source": [
        "rgb values range from 0 to 255. normalize the data by dividing every value by 255."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "OGMC2nHbOfZh",
        "scrolled": true
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "1.0 0.0\n",
            "1.0 0.0\n"
          ]
        }
      ],
      "source": [
        "X_train_flattened = X_train_flattened / 255 # devide X_train by 255\n",
        "X_val_flattened = X_val_flattened / 255 # devide X_val by 255\n",
        "print(X_train_flattened.max(), X_train_flattened.min())\n",
        "print(X_val_flattened.max(), X_val_flattened.min())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NJNdvzNHOfZh"
      },
      "source": [
        "### 3.3. Plot Function\n",
        "This function get a history of training (loss, acc, val_loss, val_acc) as input and plots it."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "4Q35JDeqOfZh",
        "scrolled": true
      },
      "outputs": [],
      "source": [
        "def plot_history(history):\n",
        "    losses, accs, val_losses, val_accs = history\n",
        "    x = [i for i in range(len(losses))]\n",
        "    \n",
        "    # plot for losses\n",
        "    plt.plot(x, losses, '-g', label='train')\n",
        "    plt.plot(x, val_losses, '-r', label='validation')\n",
        "    plt.xlabel('epoch')\n",
        "    plt.ylabel('loss')\n",
        "    plt.legend(loc='upper right')\n",
        "    plt.show()\n",
        "    \n",
        "    # plot for accuracies\n",
        "    plt.plot(x, accs, '-g', label='train')\n",
        "    plt.plot(x, val_accs, '-r', label='validation')\n",
        "    plt.xlabel('epoch')\n",
        "    plt.ylabel('accuracy')\n",
        "    plt.legend(loc='upper left')\n",
        "    plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q3XG5NOyOfZi"
      },
      "source": [
        "# 2. Run the model\n",
        "\n",
        "model specification:\n",
        "\n",
        "\n",
        "1.   1 hidden layer with 100 neurons (use `Linear` module)\n",
        "2.   one of the activation functions (`ReLU` or `sigmoid`)\n",
        "3.   a 10 neurons output layer (use `Linear` module)\n",
        "4.   softmax corss entropy loss (use `SoftmaxCrossentropy` module)\n",
        "\n",
        "\n",
        "\n",
        "We train the model for 10 epochs and `batch_size = 1024`. We also plot the history of training.\n",
        "\n",
        "You can use `model.add(Module)` to add a module to the model. The modules are added in order.\n",
        "It's also recommended to read the `model.py` file."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 44,
      "metadata": {
        "id": "t_1Mw7xYOfZj",
        "scrolled": true
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch 1: 9.773753669130038\n",
            "10.340940143563547\n",
            "10.118451431483177\n",
            "8.798582289659409\n",
            "9.380750276690554\n",
            "8.989313212494128\n",
            "8.298586797177006\n",
            "7.517557990846282\n",
            "6.778256482206764\n",
            "6.494503015615784\n",
            "6.439674643815243\n",
            "6.624626610722698\n",
            "6.440059580906958\n",
            "6.49798250972902\n",
            "6.531362365581115\n",
            "6.703882129824015\n",
            "6.846993803006413\n",
            "6.948890461815528\n",
            "6.263879459568166\n",
            "5.79500187822349\n",
            "5.767011069278679\n",
            "5.832086718543042\n",
            "5.837289265636425\n",
            "5.66524478354263\n",
            "5.714923296941416\n",
            "5.762929128932143\n",
            "5.690777567634722\n",
            "5.847768659087475\n",
            "5.721312825396405\n",
            "5.7309855326433965\n",
            "5.733252302580684\n",
            "5.87802769412468\n",
            "5.6523544598247994\n",
            "5.7016264119239795\n",
            "5.707807811183254\n",
            "5.624011833098351\n",
            "5.758515766338772\n",
            "5.720152724996394\n",
            "5.7293196610762465\n",
            "5.679523305624368\n",
            "5.711268404346155\n",
            "5.726204744090029\n",
            "5.81460995838398\n",
            "5.672213514407556\n",
            "5.768785092738076\n",
            "5.763008172884487\n",
            "5.649662212423996\n",
            "5.5637051161430495\n",
            "5.696703962086471\n",
            "5.604099884109411\n",
            "5.598827919330779\n",
            "5.5613765991093675\n",
            "5.690827568075293\n",
            "5.8141653105204005\n",
            "5.641580310942301\n",
            "5.693954243486857\n",
            "5.605879237826049\n",
            "loss = 6.5002, acc = 0.1000, val_loss = 5.6564, val_acc = 0.0984\n",
            "Epoch 2: 5.577505314969419\n",
            "5.5700788153641305\n",
            "5.7126759611757\n",
            "5.782860559332535\n",
            "5.585836721847632\n",
            "5.552744719640621\n",
            "5.653131650936901\n",
            "5.5292613952269\n",
            "5.5704846530811825\n",
            "5.548020790797622\n",
            "5.66803522385354\n",
            "5.52385126484941\n",
            "5.312303515323391\n",
            "5.3287036935525105\n",
            "5.429604426998291\n",
            "5.377229918518944\n",
            "5.29246326958647\n",
            "5.248754364173211\n",
            "5.258094393906005\n",
            "5.326125321916725\n",
            "5.4288513129592\n",
            "5.401183988453132\n",
            "5.235308533203572\n",
            "5.372757394708252\n",
            "5.331982022305835\n",
            "5.262685497396966\n",
            "5.376995217696832\n",
            "5.143332248601985\n",
            "4.876510007727835\n",
            "4.718293420019885\n",
            "4.757725255547614\n",
            "4.665043168744257\n",
            "4.625620009505141\n",
            "4.548181835037296\n",
            "4.651260972126182\n",
            "4.658513523479062\n",
            "4.520303069285935\n",
            "4.493759810970376\n",
            "4.552542759545587\n",
            "4.714494525688356\n",
            "4.528390427135944\n",
            "4.5938840940744665\n",
            "4.534222350135206\n",
            "4.453191400470078\n",
            "4.541690325427524\n",
            "4.459492614821055\n",
            "4.527483580965055\n",
            "4.535937982360052\n",
            "4.526635114307715\n",
            "4.540701622465115\n",
            "4.504854151159277\n",
            "4.404802959538974\n",
            "4.517606305277802\n",
            "4.5720856100406255\n",
            "4.534149323573439\n",
            "4.409518128905811\n",
            "4.527051146092195\n",
            "loss = 5.0908, acc = 0.0963, val_loss = 4.5042, val_acc = 0.0981\n",
            "Epoch 3: 4.448981935917964\n",
            "4.491880397327126\n",
            "4.316638443051108\n",
            "4.228773667810925\n",
            "3.458362361128903\n",
            "3.492235396743732\n",
            "3.5526729207158962\n",
            "3.3675015509703448\n",
            "3.3875141198012493\n",
            "3.390596792322092\n",
            "3.4224561870814334\n",
            "3.438230314473979\n",
            "3.4030123802285757\n",
            "3.441502217637403\n",
            "3.3817383548934545\n",
            "3.490388131627797\n",
            "3.467573125093667\n",
            "3.4046011061768313\n",
            "3.373639379988855\n",
            "3.421428223000179\n",
            "3.3931010606394936\n",
            "3.3576524099467204\n",
            "3.3633414353660336\n",
            "3.410885161935849\n",
            "3.4928413120353623\n",
            "3.3851393009106054\n",
            "3.4023568154538237\n",
            "3.444712979386092\n",
            "3.4518493332865465\n",
            "3.4450365044334403\n",
            "3.3970425344604536\n",
            "3.330009706183275\n",
            "3.3303527156303856\n",
            "3.38808778377596\n",
            "3.4378982202422326\n",
            "3.414631407581627\n",
            "3.419229041392902\n",
            "3.360802948464935\n",
            "3.3519111564732365\n",
            "3.465012127843867\n",
            "3.3782953524810058\n",
            "3.3421464780278924\n",
            "3.403415726936431\n",
            "3.3871752617287005\n",
            "3.3695817857404347\n",
            "3.3551749453680655\n",
            "3.4009226199338256\n",
            "3.381253875182829\n",
            "3.448915329037048\n",
            "3.309405943482202\n",
            "3.4295926174688853\n",
            "3.3570413286868277\n",
            "3.4155043599606665\n",
            "3.466978757806424\n",
            "3.3633281959817882\n",
            "3.3983436114386643\n",
            "3.3033454898024517\n",
            "loss = 3.4883, acc = 0.1007, val_loss = 3.3881, val_acc = 0.0955\n",
            "Epoch 4: 3.293186071014351\n",
            "3.418508519830647\n",
            "3.3825542269909508\n",
            "3.4351042327732433\n",
            "3.322836943950323\n",
            "3.4153541685330593\n",
            "3.3327016178361055\n",
            "3.401028889131547\n",
            "3.2740384670851364\n",
            "3.3633937145207233\n",
            "3.359874813537539\n",
            "3.275229141124975\n",
            "3.3641069315689442\n",
            "3.3959240544081233\n",
            "3.297516306137382\n",
            "3.394313537465682\n",
            "3.430557689204045\n",
            "3.3530073433398537\n",
            "3.3871994847519566\n",
            "3.2799744887703373\n",
            "3.34757237336441\n",
            "3.3427017589118284\n",
            "3.3334799893804634\n",
            "3.363808585527432\n",
            "3.316603541626292\n",
            "3.35811584300163\n",
            "3.400954690772406\n",
            "3.4846292002055765\n",
            "3.802312506735617\n",
            "4.089382769608443\n",
            "3.9564988000043955\n",
            "4.0334788108394335\n",
            "3.995505946168637\n",
            "4.06653286336475\n",
            "4.035114911213137\n",
            "3.925669392532283\n",
            "4.0181504541268795\n",
            "3.983905105347267\n",
            "4.053877205208156\n",
            "4.040799684802721\n",
            "3.995747862215966\n",
            "4.037324616569394\n",
            "3.8726468629703\n",
            "3.9689919405673098\n",
            "4.065214808903381\n",
            "3.992297559146178\n",
            "4.01018331981947\n",
            "4.004854893739461\n",
            "3.926762834660275\n",
            "3.98760199494578\n",
            "3.9768391028568106\n",
            "3.967890183651829\n",
            "3.9959872575964983\n",
            "4.016940842906397\n",
            "4.052443491318599\n",
            "3.8657956697610993\n",
            "3.9706953987458284\n",
            "loss = 3.6265, acc = 0.1006, val_loss = 3.9734, val_acc = 0.1048\n",
            "Epoch 5: 3.975004324985978\n",
            "3.9236142480114493\n",
            "4.002806663000811\n",
            "3.9314177873808154\n",
            "3.9686976301665124\n",
            "3.9541659589301146\n",
            "3.9994990543919364\n",
            "3.8821747442306798\n",
            "3.951897816324315\n",
            "4.050259943999747\n",
            "3.8749330283220598\n",
            "3.9400682215194376\n",
            "3.8873821067347643\n",
            "3.985006052766251\n",
            "3.9547049235008918\n",
            "3.934942686047874\n",
            "3.929348501920776\n",
            "3.9725241598752685\n",
            "3.9280412443595365\n",
            "3.9773158772460944\n",
            "3.950360755238142\n",
            "3.9594455346569766\n",
            "4.030274334560013\n",
            "4.024378281861884\n",
            "3.954156326090056\n",
            "3.913147031317211\n",
            "3.9132415343778044\n",
            "3.9874271052598074\n",
            "3.9894508648414826\n",
            "4.0366152520925835\n",
            "3.864661259390176\n",
            "3.8847694986328207\n",
            "3.9642971426318065\n",
            "3.993922695486753\n",
            "3.874716846246796\n",
            "3.90031914498425\n",
            "3.9248864763537856\n",
            "3.986717108026072\n",
            "3.875198771390107\n",
            "3.9280000821686727\n",
            "4.004750869144876\n",
            "3.775627086974132\n",
            "3.944644558016278\n",
            "3.9787799134974478\n",
            "3.9365750900829237\n",
            "3.8826642182928763\n",
            "3.868487098138271\n",
            "3.848461437607693\n",
            "3.958521521396584\n",
            "3.939517236576987\n",
            "3.856198289811217\n",
            "3.8947740222639955\n",
            "3.9585914604092407\n",
            "3.9624176926053067\n",
            "3.897515375052177\n",
            "3.96365071706773\n",
            "3.927269586760305\n",
            "loss = 3.9421, acc = 0.1005, val_loss = 3.9287, val_acc = 0.0949\n",
            "Epoch 6: 3.9423093105058653\n",
            "3.9174698341725165\n",
            "3.850847665301862\n",
            "3.9505170895533652\n",
            "3.9388707357940973\n",
            "3.925494763657758\n",
            "3.865300482754339\n",
            "3.8628833116356973\n",
            "3.905657803818526\n",
            "3.945029053198865\n",
            "3.98561434572445\n",
            "3.8268393793284927\n",
            "3.8312883072334403\n",
            "3.858498102739806\n",
            "3.9188058168079714\n",
            "3.8977325166474315\n",
            "3.9312823663866974\n",
            "3.904046878839598\n",
            "3.89693624178277\n",
            "3.8044381342136764\n",
            "3.8131288295323054\n",
            "3.899718890088974\n",
            "3.9061632272047264\n",
            "3.8224532556652857\n",
            "3.8575677509159347\n",
            "3.8302126317110794\n",
            "3.927547776878746\n",
            "3.8492506122651076\n",
            "3.898294495655873\n",
            "3.8512517012579313\n",
            "3.795470302834978\n",
            "3.911865160129556\n",
            "3.752047031729943\n",
            "3.861288376593067\n",
            "3.871779545937102\n",
            "3.8188198029666633\n",
            "3.843782825954054\n",
            "3.894290748277095\n",
            "3.955270538156208\n",
            "3.845189334825334\n",
            "3.8458121727786247\n",
            "3.7878713854475974\n",
            "3.7479417717834607\n",
            "3.931302548453062\n",
            "3.84542280218315\n",
            "3.8306295968552817\n",
            "3.933557488947394\n",
            "3.8282649843966423\n",
            "3.7940243388972914\n",
            "3.816091579358394\n",
            "3.794440244172253\n",
            "3.757176209035786\n",
            "3.824604965378226\n",
            "3.8346916968980094\n",
            "3.882143881751326\n",
            "3.890576550078836\n",
            "3.849375532164234\n",
            "loss = 3.8733, acc = 0.0996, val_loss = 3.8270, val_acc = 0.0996\n",
            "Epoch 7: 3.8534867657215006\n",
            "3.792825633550736\n",
            "3.843841437859777\n",
            "3.813318021867586\n",
            "3.821622894708952\n",
            "3.856337953716026\n",
            "3.758434474599341\n",
            "3.840546080853084\n",
            "3.9464403765446736\n",
            "3.8136692056922543\n",
            "3.8537935605046028\n",
            "3.8748598117764246\n",
            "3.826639186386115\n",
            "3.807044962000713\n",
            "3.7823175339626194\n",
            "3.866675727778395\n",
            "3.8341779392883\n",
            "3.797900522992496\n",
            "3.82388836636905\n",
            "3.824216171774225\n",
            "3.829316195676859\n",
            "3.7638060384600216\n",
            "3.7813011845355864\n",
            "3.763108181349858\n",
            "3.7609197273947794\n",
            "3.767848057704765\n",
            "3.7884681824089017\n",
            "3.888040199592152\n",
            "3.799288890512994\n",
            "3.7861378649920328\n",
            "3.8513182076964814\n",
            "3.760689315109942\n",
            "3.776795346749916\n",
            "3.784992275099454\n",
            "3.8091909351828033\n",
            "3.8337559670772325\n",
            "3.848228002636758\n",
            "3.794582228398873\n",
            "3.7804976432184345\n",
            "3.7792883454470854\n",
            "3.739739243914505\n",
            "3.752598071908707\n",
            "3.805726704548528\n",
            "3.6703445656198266\n",
            "3.8574987170986352\n",
            "3.75731106006722\n",
            "3.7936471958482523\n",
            "3.828986525344509\n",
            "3.760926950414745\n",
            "3.79467097275979\n",
            "3.830942732286114\n",
            "3.7987303095022433\n",
            "3.773887067060559\n",
            "3.678883380050544\n",
            "3.728493166641715\n",
            "3.8737978820268744\n",
            "3.6275391405586577\n",
            "loss = 3.8080, acc = 0.0987, val_loss = 3.7631, val_acc = 0.1019\n",
            "Epoch 8: 3.6907903342430037\n",
            "3.78489834534468\n",
            "3.8306622521208302\n",
            "3.788421939039523\n",
            "3.8433771796051968\n",
            "3.814425622529927\n",
            "3.765745141417228\n",
            "3.717018683370256\n",
            "3.799134040680544\n",
            "3.577272027505664\n",
            "3.816444417013627\n",
            "3.7461221723751237\n",
            "3.682852514439239\n",
            "3.697204422437107\n",
            "3.9080509402140877\n",
            "3.7667940001643863\n",
            "3.784415952362699\n",
            "3.776393475448391\n",
            "3.80170437596114\n",
            "3.786005287482042\n",
            "3.802074749666647\n",
            "3.7016359365608933\n",
            "3.7246979657074846\n",
            "3.693915030007501\n",
            "3.810047401652601\n",
            "3.779233260986935\n",
            "3.7951899199453147\n",
            "3.7465964321826077\n",
            "3.779967155957771\n",
            "3.7820703246801157\n",
            "3.6927866600575574\n",
            "3.7132500722378996\n",
            "3.7479285772101356\n",
            "3.7350640323305506\n",
            "3.7735541495989313\n",
            "3.6590230295350112\n",
            "3.725783657035742\n",
            "3.716246956119356\n",
            "3.684564093612147\n",
            "3.707681605637105\n",
            "3.64383244691411\n",
            "3.798177053120055\n",
            "3.7469664629778157\n",
            "3.7917719496185542\n",
            "3.6575190485707187\n",
            "3.6469498883646683\n",
            "3.7190371342560993\n",
            "3.611812268521582\n",
            "3.7114334334533865\n",
            "3.7103917116534513\n",
            "3.672118107392611\n",
            "3.6762314145454846\n",
            "3.7224783897190514\n",
            "3.7496081213036114\n",
            "3.7454745297308776\n",
            "3.7918650577326716\n",
            "3.77686013527306\n",
            "loss = 3.7451, acc = 0.0998, val_loss = 3.7285, val_acc = 0.0967\n",
            "Epoch 9: 3.697631574302669\n",
            "3.6952482241822775\n",
            "3.785214038382258\n",
            "3.70589605706516\n",
            "3.772537164119153\n",
            "3.689020590691689\n",
            "3.684472699410736\n",
            "3.7273901235118605\n",
            "3.7423337183485486\n",
            "3.702372281071031\n",
            "3.796476869705752\n",
            "3.7340260072539317\n",
            "3.7549566279429314\n",
            "3.669261218937563\n",
            "3.748267307882732\n",
            "3.6982337201388753\n",
            "3.765883249080723\n",
            "3.6057468866447167\n",
            "3.651128325922242\n",
            "3.769938874793824\n",
            "3.6616444877235534\n",
            "3.7227150569015106\n",
            "3.7303563884867974\n",
            "3.683804035051671\n",
            "3.673408642423791\n",
            "3.6854997276006047\n",
            "3.664480407846507\n",
            "3.6437854586973235\n",
            "3.6897427839796615\n",
            "3.7274922394657946\n",
            "3.73867655730026\n",
            "3.706696242525176\n",
            "3.65478255746401\n",
            "3.6599641662301163\n",
            "3.656510681099137\n",
            "3.7127501815040898\n",
            "3.7140531997114543\n",
            "3.6161294092423644\n",
            "3.671888648297273\n",
            "3.569379278024549\n",
            "3.5967820483977437\n",
            "3.6119815914205597\n",
            "3.7155516234065056\n",
            "3.675416275137338\n",
            "3.6634006249175366\n",
            "3.618303257524171\n",
            "3.5561600866842324\n",
            "3.640242224948029\n",
            "3.667743381429238\n",
            "3.683132615045594\n",
            "3.705322762152483\n",
            "3.626764365768958\n",
            "3.6708769441690663\n",
            "3.6549772781142176\n",
            "3.579662285641668\n",
            "3.589953818139852\n",
            "3.6319956735174608\n",
            "loss = 3.6887, acc = 0.0985, val_loss = 3.6456, val_acc = 0.0987\n",
            "Epoch 10: 3.692130547068119\n",
            "3.6290255888622895\n",
            "3.58655753063732\n",
            "3.6440463375171506\n",
            "3.609136899301073\n",
            "3.6626482286054554\n",
            "3.519737876880642\n",
            "3.6408860313158558\n",
            "3.6376692119185465\n",
            "3.6375666602222543\n",
            "3.6845291258935626\n",
            "3.6480055947234797\n",
            "3.6444595195282194\n",
            "3.575666794071025\n",
            "3.588563734897746\n",
            "3.6348956943911594\n",
            "3.6359724153196256\n",
            "3.584040989023593\n",
            "3.6740578828389245\n",
            "3.588955279644925\n",
            "3.634642428706907\n",
            "3.66218813118681\n",
            "3.622837944860763\n",
            "3.602611747139834\n",
            "3.5828861669321954\n",
            "3.5963238493324896\n",
            "3.699994256343924\n",
            "3.605765667294047\n",
            "3.630209594441661\n",
            "3.66433754202434\n",
            "3.6191491468704022\n",
            "3.608279115406217\n",
            "3.577214365970727\n",
            "3.6944027237207915\n",
            "3.6118216622497137\n",
            "3.581604470800816\n",
            "3.62573914062601\n",
            "3.591340568661865\n",
            "3.583420407236347\n",
            "3.6478732088235963\n",
            "3.552312036748691\n",
            "3.65152456693753\n",
            "3.600225301184998\n",
            "3.625630515070542\n",
            "3.630781140969333\n",
            "3.591739472624544\n",
            "3.531684034124239\n",
            "3.5793093842377517\n",
            "3.6045241100845535\n",
            "3.645033277208728\n",
            "3.6365791634188795\n",
            "3.577469105415193\n",
            "3.6082035082784993\n",
            "3.619734017878074\n",
            "3.5764411763487938\n",
            "3.532210260235493\n",
            "3.6371570489000504\n",
            "loss = 3.6193, acc = 0.1019, val_loss = 3.6042, val_acc = 0.1011\n"
          ]
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAy/0lEQVR4nO3deVxWdfr/8dfFouyLiIqigmYuKCKR2TiZS5lbtklZWcH8JrXdFrX6Nk3NTM3UVNNiY2PNaE5Oi5blpC0zWZmTWa64YZqA4gokuOAKn98f5wYBAVnuwwHu6/l43A9u7rNwcWe87/M551wfMcaglFLKc3k5XYBSSilnaRAopZSH0yBQSikPp0GglFIeToNAKaU8nI/TBdRW69atTUxMjNNlKKVUk7J69epcY0xkZcuaXBDExMSwatUqp8tQSqkmRUSyqlqmQ0NKKeXhNAiUUsrDaRAopZSHa3LnCJRSzcupU6fIzs7m+PHjTpfSLPj5+REdHY2vr2+Nt9EgUEo5Kjs7m+DgYGJiYhARp8tp0owx5OXlkZ2dTWxsbI23s3VoSETCRGSBiKSLyBYRubjC8sEiUiAi61yPx+2sRynV+Bw/fpyIiAgNATcQESIiImp9dGX3EcFLwKfGmHEi0gIIqGSdb4wxY2yuQynViGkIuE9d3kvbjghEJAQYBPwdwBhz0hiTb9fPO5f03HTu//R+ThaddKoEpZRqlOwcGuoC5ACzRWStiLwhIoGVrHexiKwXkU9EJK6yHYnIRBFZJSKrcnJy6lTMjoM7eHHliyz+cXGdtldKNU/5+fn89a9/rfV2o0aNIj8/3/0FOcDOIPABEoGZxph+wFHg4QrrrAE6G2P6Aq8AH1a2I2PMLGNMkjEmKTKy0jukz2l41+FEBUUxe93sOm2vlGqeqgqCoqKiardbsmQJYWFhNlXVsOwMgmwg2xiz0vX9AqxgKGWMOWSMOeJ6vgTwFZHWdhTj4+XDLfG3sGTbEvYf2W/Hj1BKNUEPP/wwP/30EwkJCVx44YUMGTKEm266iT59+gBw9dVXc8EFFxAXF8esWbNKt4uJiSE3N5fMzEx69uzJ7bffTlxcHMOHD+fYsWNO/Tp1YtvJYmPMPhHZJSLdjTFbgWHA5rLriEg7YL8xxohIf6xgyrOrptR+qTz77bO8lfYWD/7iQbt+jFKqjqZ8OoV1+9a5dZ8J7RJ4ccSLVS7/05/+xMaNG1m3bh1fffUVo0ePZuPGjaWXX/7jH/+gVatWHDt2jAsvvJDrrruOiIiIcvvYtm0bb7/9Nq+//jrXX38977//PhMmTHDr72Enu+8svgeYJyJpQALwtIhMFpHJruXjgI0ish54GRhvbJxEuUfrHgyIHsDsdbPRuZqVUpXp379/uWvwX375Zfr27cuAAQPYtWsX27ZtO2ub2NhYEhISALjgggvIzMxsoGrdw9bLR40x64CkCi+/Vmb5DGCGnTVUlJqQyqSPJ7Fqzyou7HBhQ/5opdQ5VPfJvaEEBp65puWrr77iv//9LytWrCAgIIDBgwdXeo1+y5YtS597e3s3uaEhj+s1dEPcDfj5+DFn3RynS1FKNQLBwcEcPny40mUFBQWEh4cTEBBAeno63333XQNX1zA8LghC/UK5tue1/Gvjvzh+WnubKOXpIiIiGDhwIL1792bq1Knllo0YMYLTp08THx/Pb37zGwYMGOBQlfaSpjZWnpSUZOo7Mc1/d/yXy/95Oe9c9w439L7BTZUppepiy5Yt9OzZ0+kympXK3lMRWW2MqThUD3jgEQHAkJghdAzpyJz1c5wuRSmlHOeRQeDt5c1tfW/j858+Z/eh3U6Xo5RSjvLIIABISUih2BQzd/1cp0tRSilHeWwQdG3VlUGdB+k9BUopj+exQQCQ0jeFbT9vY0X2CqdLUUopx3h0ECTHJRPoG8jstdqITinluTw6CIJaBJEcl8y7m97l6MmjTpejlGoCgoKCANizZw/jxo2rdJ3BgwdzrsvcX3zxRQoLC0u/d7KttUcHAVjDQ4dPHmZh+kKnS1FKNSHt27dnwYIFdd6+YhA42dba44NgUOdBdAnvovMUKOWhpk+fXm4+gieeeIInn3ySYcOGkZiYSJ8+ffjoo4/O2i4zM5PevXsDcOzYMcaPH098fDw33HBDuV5Dd9xxB0lJScTFxfHb3/4WsBrZ7dmzhyFDhjBkyBDgTFtrgBdeeIHevXvTu3dvXnzxxdKfZ1e7a7vnLG70RISUvik8/tXjZOZnEhMW43RJSnmuKVNg3Tr37jMhAVx/TCszfvx4pkyZwp133gnAe++9x6effsr9999PSEgIubm5DBgwgLFjx1Y5H/DMmTMJCAggLS2NtLQ0EhPPTL3y1FNP0apVK4qKihg2bBhpaWnce++9vPDCC3z55Ze0bl1+CpbVq1cze/ZsVq5ciTGGiy66iEsvvZTw8HDb2l17/BEBwG0JtyGI3lOglAfq168fBw4cYM+ePaxfv57w8HCioqJ49NFHiY+P57LLLmP37t3s31/1hFbLli0r/YMcHx9PfHx86bL33nuPxMRE+vXrx6ZNm9i8eXNVuwFg+fLlXHPNNQQGBhIUFMS1117LN998A9jX7trjjwgAOoV2YmjsUOasm8Njgx7DSzQflXJENZ/c7TRu3DgWLFjAvn37GD9+PPPmzSMnJ4fVq1fj6+tLTExMpe2ny6rsaCEjI4PnnnuOH374gfDwcFJSUs65n+rua7Kr3bX+xXNJTUglIz+DZVnLnC5FKdXAxo8fzzvvvMOCBQsYN24cBQUFtGnTBl9fX7788kuysrKq3X7QoEHMmzcPgI0bN5KWlgbAoUOHCAwMJDQ0lP379/PJJ5+UblNV++tBgwbx4YcfUlhYyNGjR1m4cCGXXHKJG3/bs2kQuFzT8xpCWoboSWOlPFBcXByHDx+mQ4cOREVFcfPNN7Nq1SqSkpKYN28ePXr0qHb7O+64gyNHjhAfH8+zzz5L//79Aejbty/9+vUjLi6OX/3qVwwcOLB0m4kTJzJy5MjSk8UlEhMTSUlJoX///lx00UX8+te/pl+/fu7/pcvwyDbUVZn474nM2zCPfQ/uI7hlsC0/QylVnrahdj9tQ10PqQmpFJ4qZP7m+U6XopRSDUaDoIwB0QPoHtFdh4eUUh5Fg6AMESElIYXlO5ez/eftTpejlMdoakPUjVld3ksNggpu7XsrXuKlk9sr1UD8/PzIy8vTMHADYwx5eXn4+fnVaju9j6CC9sHtuaLrFby5/k2eHPwk3l7eTpekVLMWHR1NdnY2OTk5TpfSLPj5+REdHV2rbWwNAhEJA94AegMG+JUxZkWZ5QK8BIwCCoEUY8waO2uqidSEVK5fcD1LM5ZyedfLnS5HqWbN19eX2NhYp8vwaHYPDb0EfGqM6QH0BbZUWD4S6OZ6TARm2lxPjVzZ/UrC/cL1pLFSyiPYFgQiEgIMAv4OYIw5aYzJr7DaVcBcY/kOCBORKLtqqik/Hz9u6nMTC9MXkn883+lylFLKVnYeEXQBcoDZIrJWRN4QkcAK63QAdpX5Ptv1WjkiMlFEVonIqoYaR0xNSOX46eO8s/GdBvl5SinlFDuDwAdIBGYaY/oBR4GHK6xTWU/Xsy4dMMbMMsYkGWOSIiMj3V9pJRKjEundprdePaSUavbsDIJsINsYs9L1/QKsYKi4Tscy30cDe2ysqcZEhNSEVFbuXsmWnIqnNpRSqvmwLQiMMfuAXSLS3fXSMKBiI+5FwK1iGQAUGGP22lVTbU2In4CPl4+eNFZKNWt2XzV0DzBPRNKABOBpEZksIpNdy5cAO4DtwOvAnTbXUyttAtswutto/pn2T04Xn3a6HKWUsoWt9xEYY9YBFbvdvVZmuQHusrOG+kpJSOGjrR/x2fbPGH3+aKfLUUopt9MWE+cwuttoIgMidXhIKdVsaRCcg6+3LxPiJ7Bo6yJyC3OdLkcppdxOg6AGUhJSOFV8irc3vO10KUop5XYaBDUQ3zaexKhEHR5SSjVLGgQ1lJqQytp9a1m/b73TpSillFtpENTQjb1vpIV3C73TWCnV7GgQ1FBEQARju4/lrQ1vcbLopNPlKKWU22gQ1EJqQiq5hbks/nGx06UopZTbaBDUwvCuw4kKimLO+jlOl6KUUm6jQVALPl4+3BJ/C4t/XMz+I/udLkcppdxCg6CWUvulUmSKeCvtLadLUUopt9AgqKUerXswIHoAs9fNxmqVpJRSTZsGQR2kJqSyKWcTq/eudroUpZSqNw2COrgh7gb8fPyYvVbvNFZKNX2eFQTHjrllN6F+oVzb81r+tfFfHD993C37VEopp3hOEHzwAbRvD9nZbtldakIq+cfzWbR1kVv2p5RSTvGcIEhMhMOH4S9/ccvuhsQMoWNIR21Ep5Rq8jwnCGJi4IYbYNYsOHiw3rvz9vLmtr638flPn7P70O7616eUUg7xnCAAmDoVjhyB114797o1kJKQQrEp5p9p/3TL/pRSygmeFQQJCXDFFfDSS3C8/id5u7bqyqDOg/SeAqVUk+ZZQQAwbRrs3w9z57pldyl9U/gx70dWZK9wy/6UUqqheV4QDBkCF1wAzz0HRUX13l1yXDKBvoF6T4FSqsnyvCAQgenTYds2+Oijeu8uqEUQyXHJvLvpXQpPFbqhQKWUali2BoGIZIrIBhFZJyKrKlk+WEQKXMvXicjjdtZT6tproUsXeOYZcMPYfkrfFA6fPMwHWz5wQ3FKKdWwGuKIYIgxJsEYk1TF8m9cyxOMMb9rgHrA2xseegi+/x6WLav37gZ1HkSX8C56T4FSqknyvKGhEikpEBkJzz5b712JCCl9U1iasZSs/Kz616aUUg3I7iAwwOcislpEJlaxzsUisl5EPhGRuMpWEJGJIrJKRFbl5OS4pzJ/f7j3XliyBDZsqPfubku4DUF4c/2bbihOKaUajt1BMNAYkwiMBO4SkUEVlq8BOhtj+gKvAB9WthNjzCxjTJIxJikyMtJ91d15JwQEwJ//XO9ddQrtxNDYocxZN4diU+yG4pRSqmHYGgTGmD2urweAhUD/CssPGWOOuJ4vAXxFpLWdNZXTqhXcfju8/Tbs3Fnv3aUmpJKRn8GyrPqfd1BKqYZiWxCISKCIBJc8B4YDGyus005ExPW8v6uePLtqqtQDD1hXDr34Yr13dU3PawhpGcKcdXPqvS+llGoodh4RtAWWi8h64HtgsTHmUxGZLCKTXeuMAza61nkZGG8auldDp05w441WM7qff67XrgJ8A7gh7gbmb57P4ROH3VSgUkrZy7YgMMbsMMb0dT3ijDFPuV5/zRjzmuv5DNeyvsaYAcaYb+2qp1pTp8LRozBzZr13lZqQSuGpQuZvnu+GwpRSyn6ee/loWfHxMHIkvPxyvWcxGxA9gO4R3XV4SCnVZGgQlJg2DQ4cqHczOhEhJSGFb3Z+w/aft7upOKWUso8GQYlLL4ULL3RLM7pb+96Kl3jpUYFSqknQIChR0oxu+3ZYuLBeu2of3J4rul7Bm+vfpKi4/h1OlVLKThoEZV19NZx3ntV2op4XL6UmpJJ9KJulGUvdU5tSStlEg6CskmZ0P/wAX31Vr11d2f1Kwv3CtRGdUqrR0yCo6NZboU2bejej8/Px46Y+N7EwfSH5x/PdU5tSStlAg6Aif3+47z749FNIS6vXrlITUjl++jjvbnzXTcUppZT7aRBU5o47IDCw3s3oEqMS6d2mtw4PKaUaNQ2CyoSHw8SJVjO6rLrPLyAipCaksnL3SrbkbHFjgUop5T4aBFW5/37rktK//KVeu5kQPwEfLx+9p0Ap1WhpEFSlY0e46SZ4/XXIq3tD1DaBbRjdbTRz0+Zyuvi0GwtUSin30CCoztSpUFgIf/1rvXaTkpDCviP7+Gz7Z24qTCml3EeDoDq9e8Po0fVuRje622giAyKZs36O+2pTSik30SA4l2nTIDcX5syp8y58vX2ZED+BRVsXkVfYsPPuKKXUuWgQnMsll8BFF9W7GV1KQgoni07yrw3/cmNxSilVfxoE51LSjG7HDnj//TrvJr5tPIlRiXpPgVKq0dEgqImxY+H88+vdjC41IZW1+9ayft96NxanlFL1o0FQEyXN6Favhi+/rPNubux9Iy28W+g9BUqpRkWDoKZuuQXatoVnnqnzLiICIhjbfSxvbXiLU0Wn3FicUkrVnQZBTfn5wZQp8PnnsG5dnXdzc5+byS3M5avMr9xVmVJK1YsGQW1MngxBQfVqRndF1ysIahHE/M3z3ViYUkrVna1BICKZIrJBRNaJyKpKlouIvCwi20UkTUQS7ayn3sLCYNIkePddyMys0y78ff258vwrWZi+UFtOKKUahRoFgYjcJyIhrj/cfxeRNSIyvIY/Y4gxJsEYk1TJspFAN9djIjCzhvt0zpQp4OUFL7xQ510k90rW4SGlVKNR0yOCXxljDgHDgUggFfiTG37+VcBcY/kOCBORKDfs1z7R0XDzzfDGG9Ydx3Uw4rwR1vDQJh0eUko5r6ZBIK6vo4DZxpj1ZV6rjgE+F5HVIjKxkuUdgF1lvs92vda4TZ1q9R6qYzO6kuGhD9I/0OEhpZTjahoEq0Xkc6wg+ExEgoHiGmw30BiTiDUEdJeIDKqwvLIwOeuOLRGZKCKrRGRVTk5ODUu2Ua9ecOWV8MorVnfSOigZHvo682s3F6eUUrVT0yD4f8DDwIXGmELAF2t4qFrGmD2urweAhUD/CqtkAx3LfB8N7KlkP7OMMUnGmKTIyMgalmyzkmZ0s+vWMqJkeOi9Te+5uTCllKqdmgbBxcBWY0y+iEwAHgMKqttARAJdRw6ISCDW+YWNFVZbBNzqOgk9ACgwxuyt1W/glIED4eKL4fnn4XTth3f8ff0Zc/4YHR5SSjmupkEwEygUkb7ANCALmHuObdoCy0VkPfA9sNgY86mITBaRya51lgA7gO3A68Cdtf0FHCNiHRVkZNS5GZ0ODymlGgMxNWiiJiJrjDGJIvI4sNsY8/eS1+wvsbykpCSzatVZtyQ4o7jYOl8QEGD1IZKanD8/49ipY0T+OZIJ8RN4bcxrNhWplFIgIquruIy/xkcEh0XkEeAWYLGIeGOdJ/BsXl7WFURr18IXX9R689LhoS06PKSUck5Ng+AG4ATW/QT7sC7xrHufheZkwgSIirJaVNdBcq9kcgpzWJa1zM2FKaVUzdQoCFx//OcBoSIyBjhujDnXOQLP0LKldbfxf/4Da9bUevOR3UYS4BugN5cppRxT0xYT12Od8E0GrgdWisg4OwtrUiZNguDgOjWjC/ANKL16qKi47lNhKqVUXdV0aOj/sO4huM0YcyvW/QC/sa+sJiY01OpM+t571lVEtZTcK5kDRw/o8JBSyhE1DQIv101hJfJqsa1nmDLFmsmsDs3oRnUbZQ0PaWtqpZQDavrH/FMR+UxEUkQkBViMdQ+AKtG+vTWL2d//DrVsgxHgG8DobqP5YIsODymlGl5NTxZPBWYB8UBfYJYxZrqdhTVJDz1kNaN79dVab5rcK5n9R/fzzc5vbChMKaWqVuPhHWPM+8aYB4wx9xtjFtpZVJPVsyeMHQszZsDRo7XadFS3Ufj7+OvVQ0qpBldtEIjIYRE5VMnjsIgcaqgim5Tp0yEvD/7xj1ptFtgikNHnj+b9Le/r8JBSqkFVGwTGmGBjTEglj2BjTEhDFdmk/OIXVkO6OjSjKxkeWr5zuU3FKaXU2fTKHztMmwZZWTC/dsM8o7uNtoaH9OohpVQD0iCww5gx1vmCZ5+FGjT1KxHYIpBR3UaxYPMCHR5SSjUYDQI7lDSjW7fOaj1RC9fHXa/DQ0qpBqVBYJebbrLuLahlMzodHlJKNTQNAru0bAn332+1p169usablQwP6dVDSqmGokFgp4kTISSk1kcFyb2S2XdkH//b9T+bClNKqTM0COwUEgJ33AELFsBPP9V4s9Hnj8bPx09vLlNKNQgNArvddx/4+NSqGV1Qi6DS4aFiU2xjcUoppUFgv6gouPVW607jWjSjS+6VzN4je/nfTh0eUkrZS4OgITz0EJw4Aa+8UuNNxpw/xhoe0quHlFI20yBoCN27w1VXWc3ojhyp0SZBLYIYed5IHR5SStlOg6ChTJ8OBw/Wqhldcq9k9hzew7e7vrWxMKWUp7M9CETEW0TWisjHlSwbLCIFIrLO9Xjc7nocM2AAXHKJ1Yzu1KkabTLm/DG09G6pVw8ppWzVEEcE9wFbqln+jTEmwfX4XQPU45xp02DnTmtu4xoIbhnMyG4jWbBlgQ4PKaVsY2sQiEg0MBp4w86f02SMGgVxcbVqRlcyPLRi1wqbi1NKeSq7jwheBKYB1X2cvVhE1ovIJyISV9kKIjJRRFaJyKqcWs4H3Kh4eVlHBWlp8MknNdrkyvOvtIaH9OohpZRNbAsCERkDHDDGVNdoZw3Q2RjTF3gF+LCylYwxs4wxScaYpMjISPcX25BuvBE6d4annqrRUUFwy2BGnDeC+Zvn6/CQUsoWdh4RDATGikgm8A4wVETeKruCMeaQMeaI6/kSwFdEWttYk/N8fa2jgm+/hWXLarSJDg8ppexkWxAYYx4xxkQbY2KA8cBSY8yEsuuISDsREdfz/q568uyqqdFITYW2beHpp2u0+pXddXhIKWWfBr+PQEQmi8hk17fjgI0ish54GRhvTC2m9Gqq/P3hgQfg889h1apzrh7SMoQrzruCBZv16iGllPs1SBAYY74yxoxxPX/NGPOa6/kMY0ycMaavMWaAMcZz7pyaPBnCwuCPf6zR6tf3up7dh3fzXfZ39tallPI4emexU0JC4J574IMPYPPmc65eOjykN5cppdxMg8BJ994LAQHwzDPnXLV0eEhvLlNKuZkGgZNat7aGiObNg8zMc66e3CuZ7EPZrMxeaX9tSimPoUHgtAceAG/vGk1neeX5V9LCu4VePaSUcisNAqd16AApKVZX0r17q1011C+UK7rq1UNKKffSIGgMpk2zOpL+5S/nXDW5VzK7Du3i+93fN0BhSilPoEHQGHTtCuPHw8yZ8PPP1a46tvtYa3hIrx5SSrmJBkFj8fDD1uxlM2ZUu1qoXyjDuw5nwZYFeMK9d0op+2kQNBZ9+sDYsfDSS+eczjK5VzI7C3bq8JBSyi00CBqTRx+1hoZmzap2tbHdx+Lr5atXDyml3EKDoDG56CIYNgyeew5OnKhytTC/MIZ3Hc78zfN1eEgpVW8aBI3No49al5HOmVPtajo8pJRyFw2CxmbIEOvI4Jln4PTpKle7qsdVOjyklHILDYLGRsQ6KsjIgHffrXK1ML8wLu96OQs269VDSqn60SBojMaMgd69rRbVxVXfQZzcK5msgix+2PNDAxanlGpuNAgaIy8veOQR2LQJ/v3vKle7qrtreEhvLlNK1YMGQWN1/fXQpYs1nWUVQz/h/uFc1uUyvXpIKVUvGgSNlY8PTJ8O338PS5dWudr1cdeTVZDFqj3nnvJSKaUqo0HQmN12G7RvX+0k96XDQ3r1kFKqjjQIGrOWLeGhh6wjgu8qn6tYh4eUUvWlQdDY3X47RERUe1SQ3CuZzPxMVu9d3YCFKaWaCw2Cxi4oCO67z7p6KC2t0lWu6nEVPl4+evWQUqpONAiagrvvtgLhT3+qdHEr/1Y6PKSUqjPbg0BEvEVkrYh8XMkyEZGXRWS7iKSJSKLd9TRJ4eFw553Wncbbt1e6SnKvZDLyM1izd00DF6eUauoa4ojgPmBLFctGAt1cj4nAzAaop2m6/37w9a1ykvure1xtDQ/p1UNKqVqyNQhEJBoYDbxRxSpXAXON5TsgTESi7KypyWrXDv7f/7O6ku7efdbiVv6tGBY7jPc2vafDQ0qpWrH7iOBFYBpQVcOcDsCuMt9nu14rR0QmisgqEVmVk5Pj9iKbjGnTrLuMn3++0sVNcnjImCrvnFZKNQwfu3YsImOAA8aY1SIyuKrVKnntrL8KxphZwCyApKQkz/2r0bkz3Hwz/O1vVofS1q3LLb66x9VM+ngS8zfP54L2FzhUZC0cOABXXAFbtkCbNhAZWbOvgYFOV65Us2JbEAADgbEiMgrwA0JE5C1jzIQy62QDHct8Hw3ssbGmpm/6dJg715rb+Pe/L7coIiCCYV2GMX/zfP447I+IVJazjcThwzBqFKSnwx13QH6+FQw5OVYwHDgAx45Vvq2/f+2Cw9//nOUYYxr3+6WUjWwLAmPMI8AjAK4jgocqhADAIuBuEXkHuAgoMMbstaumZqFnT7j2WnjlFZg6FUJCyi1O7pXM7f++nbX71pIY1Ugvwjp50vod1q2DDz+02m5X5uhRKxhKAqKyr/v3w4YN1vOqpvcMDKwyKE63bsWin1fwUsbb7GjtQ5t2XYgNi7Ue4dbXLuFd6BzWGT8fP7veEaUcZecRQaVEZDKAMeY1YAkwCtgOFAKpDV1Pk/TII/D++zBzpnWEUMbVPa5m8seTmb9pfuMMguJiq4fSf/8Ls2dXHQJg/QEPDISYmHPv1xg4cqT60MjJgexsWLvW+v7UKXyAa12PYoHd7bayPmob30YU8mHbItZGQa5rJKp9cPtyAVH2eXRINN5e3vV/f5RygDS1K0ySkpLMqlXaaZMRI6w/aJmZZw19DP/ncHYc3MG2e7Y1ruEOY6y7pF95xZqKc9o0R8rYWbCThz57kM/WLuAC72h+F3cPA327IJs2WUcpa9dCVlbp+kfbhLO7ayRbOvrzQ5vTLA37me+892LKvLW+Xr50Cu1UaUjEhscSGRDZuP5bKI8jIquNMUmVLtMgaKKWLYNLL4UZM+Cuu8oten3160z8eCJrJq6hX1Q/hwqsxFNPwWOPwQMPwHPPWdNyNqDjp4/z/LfP8/TypzHG8MgvH2HqwKmVD/n8/POZUCh5pKeXzhhnwsI41rs7Od06sCMmlPVRXnwfcoifDmeRcTCDnMLyV7cF+gZWGRKxYbEEtwxugHdAeTINgubIGLjkEti1y7rb2Ne3dFFuYS7tnmvHtIHTeHpY1c3qGtTrr8PEiTBhArz5pjULWwNa/ONi7vv0Pn46+BPX9byO54c/T+ewzrXbSWGhdT5i7dozIZGWBsePW8v9/KBPH+jXj+N9epHdJYKt7Vuy/fgeMvIzrMdB6+uRk0fK7bp1QOvSYOgc2pmYsBg6h3amc1hnOod21qBQ9aZB0Fx98ol15c2cOda4exmX//NyMvMz+fHuH50fkli4EMaNg+HDYdGicqFlt+0/b2fKp1NYvG0xPVr34OURL3N518vd9wNOn4atW8sfOaxda10FBVbgde8O/fqVPkxCAnn+pjQUdhzcUfo8Iz+DnQU7OVl0styPaeXf6qyAKHkeExZDmF+Y8/+dVaOmQdBcGQOJidZllps2gfeZk5Ulw0NrJ60loV2CczUuW2YFQEICfPFFg90DcPTkUf64/I/8+ds/08K7BU9c+gT3XHQPLbxb2P/DjbHOMZQ9cli71jpRXaJTJysYEhLOhETHjiBCsSlm/5H9ZOZnklWQRVZ+1pnnBdbzwlOF5X5kcIvgcuFQGhquI4o2gW00KDycBkFzNn++Nb/x/PnWp26XkuGh6QOn89Swp5ypLS0NBg2CqChYvtyaV8FmxhgWbF7Ag58/yK5Du5gQP4FnL3uWqOBG0LkkJ+fs8w4//njmzuqICOjb15qrunNn69Gpk/W1Q4fSIyljDHnH8soHRH4WmQWZZOVbYZF/PL/cj/b38adTaCcrLELPBERJWEQFRelVT82cBkFzVlQEvXpZn7RXry53Avbyf15OVn4WW+/e2vCfBjMy4Be/sOZe/t//rD9oNtucs5l7PrmHpRlL6du2LzNGzeCXnX5p+8+tlyNHzpx3KDnnkJlpXd5alpeXFQYlwVA2JEoeZY62Co4XVHk0kZWfddbJbF8vXzqGdjwz7BQaQ6fQTqXhER0SrfdRNHEaBM3d7Nnwq19Z5wxGjCh9edbqWUz6eBLrJq2jb7u+DVfPgQMwcCDk5VlHAr162frjCo4X8OTXT/LK968Q3CKYPwz9A5MumNS0P+EeO2ZdCJCVZT127jzzPCvLGmY6fbr8Nq1alQ+GikHRunXpB4WjJ4+ys2BnlWGx9/BeTIVuL20D25YGQ6eQTuWColNoJyL8I3T4qRHTIGjuTp6E886zbrxatqz05ZyjOUQ9H8XDv3yYPwz9Q8PUcvgwDBkCmzdb5wQuvti2H1Vsinkr7S2m/WcaB44e4PbE23lq2FO0Dmh97o2buqIi2Lu38pAo+f5I+SuT8Pev/oiiQwfrCA44cfoEuw/vZmfBTisw8rOs54fOPD92unwLkADfgNJwKAmKkpDoFNqJ6JDohjlHoyqlQeAJXnkF7r3XCoJLLil9+bK5l7GzYGfDDA+dOAGjR8NXX8FHH1nPbbJm7xruXnI3K7JXcFGHi5gxagZJ7Sv9N+6ZjIGDB6sOiaysqoefOneG6GjrCCMszJoYqeSr67kJC+NnP0OWOUjWoV2lgVE2KPYf3V9u94IQFRxVLizKBkWn0E6E+4XrUYVNNAg8QWGhdUSQlARLlpS+/LdVf2Py4sn2Dw8VF8ONN8J771V6Oau75BXm8djSx/jb6r8RGRjJM5c9w619b8VLdNbVWis7/FQxMHbvtoIkP986+qiKCISGnhUUhIdzOjSYfD8ht+Vp9vmeYLfXUTIkn5+K80gv2s/GU7s5IuUvkw1qEVQuKDqGdqRDcAeiQ6LpENKBDsEdCGkZomFRB9UFQYP3GlI2CQiw7th95BHrpGM/647ia3pew51L7mT+5vn2BUFJ64j33rNmULMhBIqKi3hjzRs8uvRRCo4XcO9F9/LE4CcI8wtz+8/yGP7+cP751qMqJT2c8vOtYCgJh+qep6dDfj4+Bw/S+tgxWgM9qtq9vz+nQoI4FuzHkQBvDvoJuS0Os893Nbu8vyajxTGWtob01rA3GBDrLu2SUCj5Gh0SXe77tkFt8fHSP281pUcEzUlBgXVYf/nl1uWkLsPmDiP7UDbpd6Xb80nqD3+A3/wGHnzQah3hZit2reDuT+5mzd41DI4ZzCsjX6F3m95u/znKBidO1C5Eyj4vKCi3q1OB/uR1jCC7fRDb2nizIfwU3wcfYoVfDoVe5Y9avMSLdkHtqg2LDiEdCGoR1EBvhPP0iMBThIbC3XfD009bn8p6WJ/Dknslc8fiO9hwYAPxbePd+zNnzbJC4JZbqpxPua72HdnHw/99mDfXv0mH4A68c907XB93vQ4LNCUtW0LbttajtkpOiG/dCunp+Kan0y49nXZbt5K0dBc3ulYzXl4UxcZyNDaa3I4R7GofyLZIb9JanWQrefyY9yNfZnxJwYmCs35ESMuQ0lAoDYoKYdEmsE2zH3rUI4LmJifHOiq44QbrslLgwNEDRD0fxaO/fJTfD/39OXZQCx98AMnJ1ixjH33kttYRp4pOMeP7GTzx9RMcO3WMh37xEI9e8qhHfXpT53DkiHUzXnp6+cePP5aflyIiwvpA1KMHJ7vGktMpgl1RgWwPLSL72D52H9rN7sOux6Hd7D2yl2JTfmZdHy8fokOi6RJ+Zq6KLuFdiA23vjaVzrJ6stjTTJkCr75qNaPrbDVWG/rmUPYc3sOWu7a45x/t119bAdCvnzW3gJtaRyzNWMo9n9zD5pzNjDxvJC+OeJHzI6oZw1aqrKIi68R3xYBITy9/lZSvL3TrVhoSdO8OPXpQ1O089vscZ/eh3WQfyi4NiKyCrNKmgRWvhgrwDTgTDq7GgWWfN5YPMBoEnmbXLujaFSZNsi4rBWb+MJM7l9xJ2uQ0+rTtU7/9r19vtY7o0AG++cYtrSN2Feziwc8fZP7m+cSGxfLSiJcYc/6YJvFJSzURBw+WDjOVe2zfXv7KqKioMwFREhIlN+SFh3O06DiZ+ZlWs0BXOOzI31FlZ9nIgMjy4VDmiKJjSEd8vRumCaMGgSf69a9h3jyrXUHbtuw/sp/2L7Tn/y75P3435Hd13++OHdZdwz4+8O23VqO0ejhx+gTPr3iep7556txzBChlh5MnrX/XFUNiy5azTlgjYt1f0bp1pQ8TEcGh4BZktzhOhvchtslB0k/t5aeCM51lTxefuSPcW7zpGNqx3BFFSUjEhsW6tVmgBoEn2rbN+jQzbRr88Y8ADHlzCPuO7GPznZvr9o+rpHXEzz9bRwJ1bB2x4+AOlmYs5YuML/hixxfkFObUfY4ApexijPVvPj3duq8iN/fsR16e9TUnB06dqnw/3t7WUXPr1pjWERSGBpIf5EOuv2F3yxNk+RzlR6+fSS/OYav8TG4AHG4JyJlhp9jwWLqEdWFUt1Fccd4Vdfp19KohT9Stm9WV9NVXrXmNw8JI7pXMXUvuYlPOptpffnnoEIwcaf0P8cUXtQqBvYf3sjRjqfXIXEpmfiYAUUFRDO86nJSEFC7rclnt6lHKbiI1v+Kp5H6LysKizEPy8gjcsYvA3Fw65ObSt4qb9Yp9fSgMDeBQUAtyA/az1y+bnb6fknvlfvi/ugVBdTQImrNHHoF33rGms3zsMa7teS13L7mb+Zvm1y4ITpyAa66xzg0sWnTO/kEHjx3kq8yvSj/1b8ndAkCYXxhDYobw0MUPMTR2KD1a99BzAKp5EIHgYOsRG1uzbYyxhp4qCQyv3FyCcnMJysujfW4u8bm5mNxcik90s6d8HRpq5q68ElassNoGBAYyeM5gDhw9wOa7Ntds+6Iiq3XE/PnWFJO33nrWKkdPHmX5zuV8kfEFSzOWsmbvGgyGAN8ALul0CcNihzE0digJ7RKadkdQpZowHRryZI8+as0L8PrrMGUKyb2SufuTu9l0YBNxbeKq39YYq5Hd/Pnw5z+XhsDJopOszF5Z+of/u+zvOFV8Cl8vXy7ueDG/vfS3DOsyjP4d+mu3SaWaAD0i8ARDhlgnj3/6iX2nDtL++fY8funjPDH4ieq3+/3v4fHHKX7wAdY8cGPpUM/yncspPFWIIFzQ/gKGxgxlWJdhDOw4kMAWDTMVpVKqdhy5akhE/IBlQEusI48FxpjfVlhnMPARkOF66QNjTLXXNmoQ1MF//mPNG/z66/DrX59zeMgYw97nn6T91Cf5clBHrhtxiIMnrcvoekX2Kh3qubTzpYT7hzfkb6KUqiOnhoZOAEONMUdExBdYLiKfGGO+q7DeN8aYMTbWoS67zGpP/ac/QUpKpcNDmfmZpZ/4/RYtYdbcfD7uBvdf5cW1541jaOxQhsYOpV1QO4d/GaWUu9kWBMY61Ci5xc7X9Wha41DNhYh1ruDaa2HBAq4bcx33fHIPT3z9BGEtw/gi4wsy8q2Dsmv2hTNn3iHyeneh9yf/ZlsHe6eZVEo5z9ZzBCLiDawGzgNeNcZMr7B8MPA+kA3sAR4yxmyqZD8TgYkAnTp1uiArK8u2mput4mLo08e6uWX9eobOHcaXmV8S2jKUwTGDGRY7jFFH29Plml8hbmwdoZRqHBy/s1hEwoCFwD3GmI1lXg8Bil3DR6OAl4wx1V4oq+cI6uGtt6x20YsWsX9If3Yf3k3ftn2tSzp/+sm6a9jX1y2tI5RSjUt1QdAgTbaNMfnAV8CICq8fMsYccT1fAviKiAfMPO6Q8eOt6Syfeoq2gW1IjEq0QmD/fquT6KlT8PnnGgJKeRjbgkBEIl1HAoiIP3AZkF5hnXbiurVURPq76smzqyaP5+NjtZtYudKaYB7OtI7YuxcWL4aePR0tUSnV8Ow8IogCvhSRNOAH4D/GmI9FZLKITHatMw7YKCLrgZeB8aap3djQ1KSkQLt21ixmJa0jNmyABQtgwACnq1NKOcDOq4bSgH6VvP5ameczgBl21aAq4ednzS08dSoMGwb/+x/MnWsdFSilPFLznohTVW7SJAgPt0LgueesE8hKKY+lvYY8UXAwzJkDe/bA5MnnXF0p1bxpEHiqsWOdrkAp1Ujo0JBSSnk4DQKllPJwGgRKKeXhNAiUUsrDaRAopZSH0yBQSikPp0GglFIeToNAKaU8XJObvF5EcoC6zkzTGsh1YzlNnb4f5en7cYa+F+U1h/ejszEmsrIFTS4I6kNEVlU1MYMn0vejPH0/ztD3orzm/n7o0JBSSnk4DQKllPJwnhYEs5wuoJHR96M8fT/O0PeivGb9fnjUOQKllFJn87QjAqWUUhVoECillIfzmCAQkREislVEtovIw07X4yQR6SgiX4rIFhHZJCL3OV2T00TEW0TWisjHTtfiNBEJE5EFIpLu+jdysdM1OUVE7nf9P7JRRN4WET+na7KDRwSBiHgDrwIjgV7AjSLSy9mqHHUaeNAY0xMYANzl4e8HwH3AFqeLaCReAj41xvQA+uKh74uIdADuBZKMMb0Bb2C8s1XZwyOCAOgPbDfG7DDGnATeAa5yuCbHGGP2GmPWuJ4fxvofvYOzVTlHRKKB0cAbTtfiNBEJAQYBfwcwxpw0xuQ7WpSzfAB/EfEBAoA9DtdjC08Jgg7ArjLfZ+PBf/jKEpEYoB+w0uFSnPQiMA0odriOxqALkAPMdg2VvSEigU4X5QRjzG7gOWAnsBcoMMZ87mxV9vCUIJBKXvP462ZFJAh4H5hijDnkdD1OEJExwAFjzGqna2kkfIBEYKYxph9wFPDIc2oiEo41chALtAcCRWSCs1XZw1OCIBvoWOb7aJrpIV5NiYgvVgjMM8Z84HQ9DhoIjBWRTKwhw6Ei8pazJTkqG8g2xpQcIS7ACgZPdBmQYYzJMcacAj4AfuFwTbbwlCD4AegmIrEi0gLrhM8ih2tyjIgI1hjwFmPMC07X4yRjzCPGmGhjTAzWv4ulxphm+amvJowx+4BdItLd9dIwYLODJTlpJzBARAJc/88Mo5meOPdxuoCGYIw5LSJ3A59hnfn/hzFmk8NlOWkgcAuwQUTWuV571BizxLmSVCNyDzDP9aFpB5DqcD2OMMasFJEFwBqsK+3W0kxbTWiLCaWU8nCeMjSklFKqChoESinl4TQIlFLKw2kQKKWUh9MgUEopD6dBoFQDEpHB2uFUNTYaBEop5eE0CJSqhIhMEJHvRWSdiPzNNV/BERF5XkTWiMgXIhLpWjdBRL4TkTQRWejqUYOInCci/xWR9a5turp2H1Sm3/88112rSjlGg0CpCkSkJ3ADMNAYkwAUATcDgcAaY0wi8DXwW9cmc4Hpxph4YEOZ1+cBrxpj+mL1qNnrer0fMAVrbowuWHd6K+UYj2gxoVQtDQMuAH5wfVj3Bw5gtal+17XOW8AHIhIKhBljvna9/iYwX0SCgQ7GmIUAxpjjAK79fW+MyXZ9vw6IAZbb/lspVQUNAqXOJsCbxphHyr0o8psK61XXn6W64Z4TZZ4Xof8fKofp0JBSZ/sCGCcibQBEpJWIdMb6/2Wca52bgOXGmALgoIhc4nr9FuBr1/wO2SJytWsfLUUkoCF/CaVqSj+JKFWBMWaziDwGfC4iXsAp4C6sSVriRGQ1UIB1HgHgNuA11x/6st06bwH+JiK/c+0juQF/DaVqTLuPKlVDInLEGBPkdB1KuZsODSmllIfTIwKllPJwekSglFIeToNAKaU8nAaBUkp5OA0CpZTycBoESinl4f4/pu8hokDQ4nEAAAAASUVORK5CYII=",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEGCAYAAABy53LJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABKM0lEQVR4nO3dd3hUZfbA8e+bECCh946AUhMQIrKKiGJhQWwQVKpdVNh1cZW1/zbY1oKKDbEAi4AoUhWxsqAoRWoivSOh9xYg7fz+eDMQQsokuXfuJDmf55knyczc954MYc685Z7XiAhKKaWUv0K8DkAppVThoolDKaVUnmjiUEoplSeaOJRSSuWJJg6llFJ5UsLrAAKhatWq0qBBA6/DUEqpQmXp0qX7RaRa5vuLReJo0KABS5Ys8ToMpZQqVIwx27K6X4eqlFJK5YkmDqWUUnmiiUMppVSeFIs5jqwkJyeTkJDAqVOnvA6lSChdujR169YlLCzM61CUUi4rtokjISGBcuXK0aBBA4wxXodTqIkIBw4cICEhgYYNG3odjlLKZcV2qOrUqVNUqVJFk4YDjDFUqVJFe29KFRPFNnEAmjQcpK+lUsVHsU4cqhASgf/+F44e9ToSpYotTRweOXz4MCNGjMjzcTfccAOHDx92PqDCIi4O7rkHxo71OhKlii1NHB7JLnGkpqbmeNysWbOoWLGiS1EVAnFx9uvKld7GoVQxVmxXVXntySefZNOmTbRu3ZqwsDDKli1LrVq1WLFiBatXr+bWW29l+/btnDp1in/84x8MGDAAOFs+5fjx43Tt2pUOHTowf/586tSpw4wZMwgPD/f4N3NZfLz9qolDKc9o4gAGfzeYFbtXONpm65qtGd5leLaPv/LKK6xcuZIVK1Ywd+5cunXrxsqVK88sZx09ejSVK1fm5MmTXHrppcTExFClSpVz2tiwYQMTJ07k448/5vbbb2fKlCn069fP0d8j6GTscYiATsorFXA6VBUk2rVrd841EO+88w4XX3wxl112Gdu3b2fDhg3nHdOwYUNat24NwCWXXMLWrVsDFK1HRGziKFXKTo4nJHgdkVLFkvY4IMeeQaCUKVPmzPdz587lp59+YsGCBURERHD11VdneY1EqVKlznwfGhrKyZMnAxKrZ/bsgf37oUcPmDoVVq2CevW8jkqpYkd7HB4pV64cx44dy/KxI0eOUKlSJSIiIli7di0LFy4McHRByjdM1aeP/arzHEp5QnscHqlSpQpXXHEFUVFRhIeHU6NGjTOPdenShZEjR9KqVSuaNm3KZZdd5mGkQcQ3MX711VCrliYOpTyiicNDn332WZb3lypVim+//TbLx3zzGFWrVmVlhjfOxx9/3PH4gk5cHNSpA1WqQFSUJg6lPKJDVarwiI+Hiy+230dGwurVkJbmbUxKFUOaOFThkJQEa9ZAq1b256goOHkStmzxNi6liiFNHKpwWLMGUlLOTRygw1VKeUAThyocfBPjvqGqFi3sV00cSgWcq4nDGNPFGLPOGLPRGPNkFo83M8YsMMacNsY8nsdjHzfGiDGmqpu/gwoS8fFQsiQ0aWJ/LlcOGjTQxKGUB1xLHMaYUOB9oCvQAuhtjGmR6WkHgUeAYXk51hhTD7ge+NOt+FWQiYuzE+IlMiwEjIzUxKGUB9zscbQDNorIZhFJAj4Hbsn4BBHZKyKLgeQ8HvsW8C9AXIs+yJQtWxaAnTt30rNnzyyfc/XVV7NkyZIc2xk+fDiJiYlnfi40ZdozrqjyiYqCdesgOfOfj1LKTW4mjjrA9gw/J6TfV6BjjTE3AztEJM6JIAub2rVrM3ny5HwfnzlxFIoy7Xv22JtvYtwnKsomjSzqeCml3ONm4siqbKm/PYQsjzXGRADPAP+XawPGDDDGLDHGLNm3b5+fpw2cJ5544pz9OGJjYxk6dCjXXnst0dHRtGzZkhkzZpx33NatW4lKX1F08uRJevXqRatWrbjjjjvOqVX18MMP07ZtWyIjI/n3v/8N2MKJO3fupFOnTnTq1AmwZdr3798PwJtvvklUVBRRUVEMHz78zPmaN2/OAw88QGRkJJ07dw58TazME+M+urJKKU+4eeV4ApCxAl1dYGcBj70QaAjEpe9xXRdYZoxpJyK7MzYgIh8BHwG0bds254Q1eDCsWOFnaH5q3RrS33yz0qtXLwYPHszAgQMBmDRpEt999x2PPvoo5cuXZ//+/Vx22WXcfPPN2e7n/cEHHxAREUF8fDzx8fFER0efeeyll16icuXKpKamcu211xIfH88jjzzCm2++yZw5c6ha9dw1BUuXLmXMmDEsWrQIEeEvf/kLV111FZUqVfK+fLsvcbRsee79zZpBSIhNHLffHrh4lCrm3OxxLAYaG2MaGmNKAr2ArwpyrIj8ISLVRaSBiDTAJpjozEmjMGjTpg179+5l586dxMXFUalSJWrVqsXTTz9Nq1atuO6669ixYwd79uzJto1ffvnlzBt4q1ataJVhKGfSpElER0fTpk0bVq1axerVq3OM59dff6V79+6UKVOGsmXL0qNHD+bNmwcEQfn2+Hhbm6patXPvL10aLrpIexxKBZhrPQ4RSTHG/A34HggFRovIKmPMQ+mPjzTG1ASWAOWBNGPMYKCFiBzN6li3Ys2pZ+Cmnj17MnnyZHbv3k2vXr2YMGEC+/btY+nSpYSFhdGgQYMsy6lnlFVvZMuWLQwbNozFixdTqVIl7r777lzbEcm+U+Z5+fa4uPOHqXy0ZpVSAefqdRwiMktEmojIhSLyUvp9I0VkZPr3u0WkroiUF5GK6d8fze7YLNpvICL73fwd3NSrVy8+//xzJk+eTM+ePTly5AjVq1cnLCyMOXPmsG3bthyP79ixIxMmTABg5cqVxKcP6Rw9epQyZcpQoUIF9uzZc07BxOzKuXfs2JHp06eTmJjIiRMnmDZtGldeeaWDv20+JSfbmlSZJ8Z9oqJg40ZbfkQpFRBaHddDkZGRHDt2jDp16lCrVi369u3LTTfdRNu2bWndujXNmjXL8fiHH36Ye+65h1atWtG6dWvatWsHwMUXX0ybNm2IjIykUaNGXHHFFWeOGTBgAF27dqVWrVrMmTPnzP3R0dHcfffdZ9q4//77adOmjfe7CvqW2+aUONLSYO1aaNMmsLEpVUyZnIYoioq2bdtK5usb1qxZQ/PmzT2KqGhy5TWdMAH69YM//ji7iurck9ryI59+Cv37O3tupYo5Y8xSEWmb+X6tVaWCW3w8hIVB06ZZP37RRfZxnedQKmA0cajgFh9vexRhYVk/HhZml+Wucm/thFLqXMU6cRSHYbpAce21zGlFlY+urFIqoIpt4ihdujQHDhzQ5OEAEeHAgQOULl3a2Yb37YNdu7KfGPeJioJt2+DoUWfPr5TKUrFdVVW3bl0SEhIIxnIkhVHp0qWpW7eus43+8Yf96k/iALts97LLnI1BKXWeYps4wsLCaNiwoddhqJzEpdexzG2oKjLSfl21ShOHUgFQbIeqVCEQHw81akD16jk/r2FDCA/XeQ6lAkQThwpe8fG5D1OBLXSomzopFTCaOFRwSkmxQ0+5DVP56MoqpQJGE4cKTuvXw+nT/vU4wCaO3bvhwAF341JKaeJQQcq3B4e/iSPjBLlSylWaOFRwiouDEiXA39pXuhugUgGjiUMFp/h4mzRKlvTv+XXqQIUKmjiUCgBNHCo4+buiyscYnSBXKkA0cajgc/AgJCT4v6LKx5c4tIyMUq7SxKGCT14nxn0iI+HQIbu6SinlGk0cKvjkN3HoBLlSAaGJQwWfuDioVg1q1szbcZo4lAoITRwq+Pgmxo3J23HVqtm6Vpo4lHKVJg4VXFJT7Rt/XoepfLRmlVKu08ShgsuGDXDqVN5XVPlERdl9OdLSnI1LKXWGJg4VXPI7Me4TFQXHj8OffzoXk1LqHJo4VHCJi4PQUP9LjWSmE+RKuU4Thwou8fHQrBnkd/9yX7FDTRxKuUYThwoueS01klmFClC3riYOpVykiUMFj0OH7NxEQRIH2OEqLa+ulGs0cajg8ccf9mt+V1T5REXBmjV2F0GllOM0cajgUdAVVT5RUXb3wE2bCh6TUuo8mjhU8IiLgypVoHbtgrWjK6uUcpWricMY08UYs84Ys9EY82QWjzczxiwwxpw2xjzuz7HGmNeNMWuNMfHGmGnGmIpu/g4qgPJbaiSz5s1tG5o4lHKFa4nDGBMKvA90BVoAvY0xLTI97SDwCDAsD8f+CESJSCtgPfCUW7+DCqCClhrJKCICGjXSCXKlXOJmj6MdsFFENotIEvA5cEvGJ4jIXhFZDCT7e6yI/CAivlnPhUBdF38HFSibNkFiYsEnxn10N0ClXONm4qgDbM/wc0L6fU4eey/wbb6iU8HFqYlxn6goWL/eTpIrpRzlZuLIaqDa3z09cz3WGPMMkAJMyLIBYwYYY5YYY5bs27fPz9Mqz8THQ0gItMg8mplPUVF2+GvdOmfaU6qQSZM0pq6ZSkqa88vS3UwcCUC9DD/XBXY6cawx5i7gRqCvSNYbTIvIRyLSVkTaVqtWLU+BKw/ExUHTphAe7kx7WnoksJ5/HgYP9joKlcGMtTOImRTD1DVTHW/bzcSxGGhsjGlojCkJ9AK+KuixxpguwBPAzSKS6ELcygsFLTWSWdOmUKKETpAHwvTp8O9/w3vvwYEDXkejsL2NoT8PpXHlxvRo3sPx9l1LHOkT2H8DvgfWAJNEZJUx5iFjzEMAxpiaxpgE4J/As8aYBGNM+eyOTW/6PaAc8KMxZoUxZqRbv4MKkCNHYOtWZxNHyZLQpIn2ONy2bRvccw9ccIEdGpwxw+uIFLa3Ebcnjuc6PkeJkBKOt+98ixmIyCxgVqb7Rmb4fjfZrIrK6tj0+y9yOEzlNadKjWQWFQVLljjbpjorORl697abZs2eDdddB1OmwL33eh1ZsZaxt9G7ZW9XzqFXjivvOb2iyicqCjZvhhMnnG1XWc89BwsWwMcfw4UXQkwM/Pij7UEqz7jd2wBNHCoYxMdDxYq2HLqTfBPkq1c7266C776DV1+FBx+E22+398XE2F7I1197G1sxFojeBmjiUMEgLs4OUxW01EhmvppVOkHurJ07oX9/aNkS3nrr7P1/+QvUqWOHq5QnAtHbAE0cymtpaXaOw+lhKrDDJ6VK6QS5k1JToW9fe5X/pEnnLp8OCYEePWxv5Phx72IspgLV2wBNHMprW7bYOQg3EkdoqL2gUBOHc158EebOhQ8+sFv8ZhYTA6dOwbda0CHQAtXbAE0cymtxcfar0yuqfLRmlXPmzIGhQ+HOO+0tKx06QPXqOlwVYGmSRuzPsTSp0sT13gZo4lBei4+3cxu+iWynRUbCjh1w+LA77RcXe/faIaomTeD997N/Xmgo3HorfPON7XmogJi+djrxe+ID0tsATRzKa/Hx0LixLYXuBp0gL7i0NNvDOHjQzmuULZvz82Ni7BzHDz8EJr5izje30aRKE3pF9QrIOTVxKG/5VlS5RXcDLLjXX4fvv4e33/ZvLqpTJ6hUSYerAiTQvQ3QxKG8dOyYvUDPjYlxn/r17SdkTRz5M38+PPOMvVZjwAD/jgkLg5tvhq++gqQkd+Mr5rzobYAmDuUl35u5m4nDGJ0gz6+DB21JkQsugI8+ytt1NjExdl5pzhzXwlPe9DZAE4fyktsrqnwiIzVx5JWIrTm1axd8/jlUqJC346+/3vb0dLjKNbn2Ng4cgBtucKVygiYO5Z34eChf3g4nuSkqCvbvtyuDlH/efddWun3tNbj00rwfX7o03HijLbmemup4eCqX3oaIHVr86SdXdsHUxKG849uDw+lSI5npBHneLF0Kjz9u5yn+8Y/8txMTA/v2wbx5zsWmAD96G2PGwNSp9oLNNm0cP78mDuWNtDSbONwepgJNHHlx9CjccQfUrGnffAqS1Lt2tSVJdLjKcTn2NjZuhEcegauvhscec+X8mjiUN7Zts6uq3JwY96lRA6pU0cSRG9/wxtatMHEiVK5csPbKlIEuXewn37Q0R0JUufQ2kpOhXz+7su3TT+0FmS7QxKG84dYeHFnxXZmuiSNnn3wCX3wBL7wAV1zhTJsxMbaa7qJFzrSncu5tvPiifa1HjoR69VyLwa/EYYyZYozpZozRRKOcERd3dqlsIERF2avHRQJzvsLmjz/s8Mb118MTTzjX7o032k+/OlzliBx7G7/9ZhPHnXfa4UYX+ZsIPgD6ABuMMa8YY7Ioi6lUHsTH27LnuZWvcEpUlB2/T0gIzPkKkxMn7AV+FSvCuHG2PLpTKlSwyWjKFE3aDsi2t3H0qB2iuuACuyLOZX79hYjITyLSF4gGtgI/GmPmG2PuMcaEuRmgKqLi4gIzTOWjE+TZ+/vfYd06mDDBzgc5LSbGzpssX+5828VIjr2Nv/8d/vzTJv7y5V2Pxe9LDY0xVYB+QH9gOTAB6ADcBVztRnAqeIgIfaf2Ze3+tUSERRAeFk54iXDCw8LtzyWy+Dn9OZmfXzYJWm7axJHbbib5xL4zj4e4ORLqq767cqVd7aOscePs6qnnnoNrrnHnHLfcYifdp0yB6Gh3zlEM+Hob47qPO7e3MWmSnQh/7jnn5qZyYcSP7qMxZirQDBgH/FdEdmV4bImItHUvxIJr27atLFmyxOswCrX52+dzxegruKzuZZQJK8PJlJOcTD5JYnLime9Pptif0yTnFTTtEmDRJ3DrHTCj+dn7S4aWzDLRZJWIqpepTs2yNalVtha1ytU6832ZkmWyP3Ht2nbYZOxYh16VQm7dOrjkEnubPRtKuFiy4rrr7DDhmjXuX7dTBKVJGm0+bMOplFOsGrjqbOLYvt323Js0gV9/tfNJDjLGLM3q/d3fv5T3ROR/WT0Q7ElDOWPUslGUCSvDD/1+oFypctk+T0RITks+J5Fk/r76ZzOAD+jT5z9cV7Nsls/N/PPhU4fZeWwnJ1NOciLpBPsS95GSlnLe+cuVLGeTSIZkUqus/b5bo1pErFjCiRP7qBJRxd0eTrA7dcrOa5QuDZ995m7SADtcNXCgLX/h1t4rRZivtzG++/izSSMtDe66yy7BnTDB8aSRE3//WpobY5aJyGEAY0wloLeIjHAtMhU0jicd54tVX3BH5B05Jg0AYwwlQ0tSMrQkFcimvtHer6FcOW6/8V/5nohNkzQOnjzIrmO72HV8F7uP7z73++O7WL5rOd8e/5ZjSccAeCMVHloNNV+rTkiJEtQsW/O85FKr3Lnf1yhTg1IlSuUrxqD22GN2gcI330CdOu6fr3t3GDTIDldp4siTbOc23njDFpEcNQouuiigMfmbOB4QkTPbfonIIWPMA4AmjmJg0qpJnEg+wX3R9znTYHw8tGxZoNU7ISaEqhFVqRpRlZY1Wub43BNJJ9h1fBcSMYqIha8wuuWzrKuUyq7ju9h1bBd/HvmTRTsWse/EPoTzh24rh1fOMrE0rNiQbk26UTK0ZL5/D09MngwjRtiyIjfcEJhz1qxpx9+nTIH/+7/AnLOImLZm2pneRmhI+gV9y5bZcvc9esA99wQ8Jn/nOOKBiyX9ycaYUCBeRArFRwed4yiYK0ZfwcGTB1k9cDWmoOPTInaTnz597JtXIP3+O/zlLzBtmt3eNJPk1GT2Je7LsRez65j9/nSqLRzXqFIjnr/6eXq37F04hr42b7a1i5o3h19+gZIBTHrDh8Ojj8KGDQH/hFxYpUkarUe25nTqaVYPXG0TR2KinZc6etR+CKtSxbXzF3SO43tgkjFmJCDAQ8B3DsangtSafWuYv30+r133WsGTBtglg0eOBHYprk+LFvbrypVZJo6w0DBql6tN7XK1c2xGRDh86jC/bf+N5+Y8R79p/Xht/mu8fM3L3ND4BmdeJzckJUGvXnZy+vPPA5s0wH46fvRR2+tw8iLDImzammn8sfePc3sbQ4bA2rV2a14Xk0ZO/P2I9ATwP+BhYBAwG/iXW0Gp4DF6+WhKhJTgzovvdKbBQJYayaxsWWjQoMD7jxtjqBReiRub3MjSAUuZGDORxOREbpx4I1eOuZJf//zVmXid9tRTsHgxjB5tX4dAq1/flmjXq8j9kuXcxjff2J76o4/aFYIe8fcCwDQR+UBEeopIjIh8KCJaZL+IS05N5tP4T7mxyY3UKOvQhWG+zZta5jwv4RqHdwMMMSH0iurF6oGrGdltJJsPbebKMVfS7bNuxO2Oc+w8BTZzJrz5Jvztb/aTv1diYmzy+vNP72IoJHy9jf/r+H+2t7F3r91cq2VLePllT2Pzt1ZVY2PMZGPMamPMZt/N7eCUt2aun8neE3u5r41Dk+JgexyNGkG5nFdnuSYqynbzHd4LOyw0jAfbPsjGRzby6nWvMn/7fFp/2Jq+U/uy6eAmR8+VZwkJdtlm69bw+uvexhITY79OneptHEHuvN6Gb0fGI0fs8unSpT2Nz9+hqjHYelUpQCfgU+zFgKoIG7V8FLXK1qLLRV2ca9S3eZNXoqIgJcVO0LogIiyCf13xLzY/spmnOjzFtDXTaPZ+MwZ+M5Bdx3bl3oDTUlLsvuFJSbbyrcdvOFx0kd2DRYercnReb+ODD+ww1WuvBa4waA78TRzhIjIbuwprm4jEAi7VJ1DBYOexnXy78Vvubn33+aWb8ysx0b5hB2LzpuxkLD3iokrhlXj52pfZ9MgmBkQP4ONlH3PhOxfy1E9PcejkIVfPfY7YWHtF8Ycf2quLg0FMjK3kunu315EEpfN6G2vW2Otu/vpXW5MqCPibOE6ll1TfYIz5mzGmO1A9t4OMMV2MMeuMMRuNMU9m8XgzY8wCY8xpY8zj/hxrjKlsjPnRGLMh/WslP38HlQdjV4wlTdK4t829zjW6apW92tXLHkezZvb6kQJOkPurVrlavN/tfdYOWkuP5j149bdXafROI1799VUSkxPdPflPP9mx8Pvus8ufg0VMjB16mTbN60iC0jm9jZRU6NvXLuwo6I6MThKRXG/ApUBZoC522GoKcFkux4QCm4BGQEkgDmiR6TnV09t+CXjcn2OB14An079/Eng1t/gvueQSyY9Ve1fJ2BVj83VsYZaWliYXvXORdBzT0dmGP/lEBEQ2bHC23bxq2lSke3dPTr1i1wrpNqGbEIvUGlZLPlj8gSSlJDl/ol27RGrUEGnRQuTECefbL6hmzUSuucbrKIJOalqqtBzRUpq820RSUlNE/vUv+39m+nRP4gGWSBbvqbn2ONIv9rtdRI6LSIKI3CN2ZdXCXA5tB2wUkc0ikgR8DtySKWntFZHFQHIejr0F8FWpGwvcmtvvkF9vL3ybAV8PYM/xPW6dIij9su0XNh7c6OykONgVVWXK2MlxLzm8siovLq55MTP7zGTePfNoVKkRD3/zMM3fb87EPybmWhzSb6mpdm+Go0dt5dSICGfadVJMDPz8M+zf73UkQeWc3sYv8+xihgcesBWGg0iuiUPssttLTN6vaqoDbM/wc0L6fQU9toakV+dN/5rlkJkxZoAxZokxZsm+ffvyFLjPY+0fIyk1ifd+fy9fxxdWo5aPonyp8vRs0dPZhh0oNeKIqCjYuBFOnvQshA71OzDvnnnM7D2TMiXL0GdqH6I/jGbWhlm+Xnf+vfKKrXb77rvBWxcqJsYmuBkzvI4kaJwzt1Hnr9C/v11M8NZbXod2Hn//By8HZhhj+htjevhuuRyTVaLx939EQY61Txb5SETaikjbatWq5eXQM5pUacItzW5hxJIRnEg6ka82Cpsjp44wefVkekf1JiLMwU+qIt6vqPKJjLTxrFnjaRjGGLo16cbyB5czoccEjiUdo9tn3bjqv1fx25+/5a/RefNsLag+fezyzWDVujU0bKirqzI409u48jlCBw6yiwcmTLC99CDjb+KoDBzArqS6Kf12Yy7HJAAZd0uvC+z083w5HbvHGFMLIP3rXj/bzJch7Ydw8ORBxqwY4+ZpgsbElRM5mXLS+WGqhAQ4dMjbFVU+vuWMAZogz02ICaFPyz6sGbSGETeMYMPBDXQY04GbJt5E/J54/xvav98uvW3UCEaODJ6J1KwYY3sdP/0Ehw871uzqfau9WfZcQBl7G72Xp9ghxqFD7ZX2wSiriQ8nbtg6WJuBhpyd4I7M5rmxnDs5nu2xwOucOzn+Wm6x5Hdy3Kf9qPbScHhDSU5NLlA7hcGlH10qLUe0lLS0NGcbnjnTTvLNm+dsu/mRlCRSsqSdeAxCx08fl5d/eVkq/KeCmFgj/ab2k00HN+V8UFqaSLdu9vdatiwwgRbUggX2b2LcuAI3dfTUUXlk1iNiYo1UeqWSzFw304EAA2fyqslCLDLtmzdFypUTufJKkZQUr8PKdnLc3yQwBhid+ebHcTcA67ErpJ5Jv+8h4KH072tiexdHgcPp35fP7tj0+6tga2VtSP9aObc4Cpo4pq6eKsQiX6z8okDtBLv43fFCLDJ8wXDnG3/5Zfvndviw823nR6tWIjfc4HUUOTqQeECe+PEJKf1iaQl7PkwGfTNIdh3blfWT33jDvr7vvhvYIAsiNVWkTh2RW28tUDMz182Uem/WExNr5MGvH5TWI1sLsciTPz5ZKD7s+VZStRjeRNLaXy5SvrzI1q1ehyUiBU8cMRlufYHJwDv+HBsMt4ImjpTUFGn8TmNp+1Fb5z+JB5F/fPsPKflCSdl/Yr/zjd9xh0iDBs63m199+ojUr+91FH7ZcXSHPPT1QxI6NFQiXoqQp396Wg6dPHT2CYsWiZQoYZcYF7a/z7//XaR0aZFjx/J86J7je6TX5F5CLNLi/Rby25+/iYhIYlKiDPhqgBCLdBzTUXYc3eF01I7y9TbiBvW0b8njx3sd0hkFShznHWTnRv6Xn2O9uBU0cYiIjFw8UohF5m6ZW+C2gtGp5FNS+dXKctuk29w5QfPmIjff7E7b+fHSS/bP/8gRryPx2/r968+8UVZ6pZK89utrkrh3p03IF1wgcvCg1yHm3dy59t9h0iS/D0lLS5Mxy8dI5VcrS8kXSsrQuUPlVPKp8543Lm6cRLwUIdVeqyY/bvrRyagd4+tt3PZYfUkLDRXp3dvrkM7hdOJoir3OwvOk4M/NicSRmJQo1V6rJt0mdCtwW8Hoi5VfCLHIdxu+c77xkydFQkJEnn3W+bbza8YM++e/YIHXkeTZsp3LpOv4rkIs8m6nspIaGiJJv/7idVj5k5IiUr267ZH6YeOBjXLt2GuFWOSKUVfI6r2rc3z+qr2rpMX7LcTEGhk6d6i9qC6ITF41Wco8hRytX8P2gA8d8jqkcxR0qOpY+jyE77YeiPHn2GC4OZE4RESGzh0qxCKr9q5ypL1g0nlcZ6n3Zj13/mMtWWL/1L780vm282vTJhvTxx97HUm+/bzpf7KnUkmZ0QSp92Y9GfH7iCw/eQe9AQNEypa1HzCykZyaLK/++qqEvxgu5V4uJx8s/kBS01L9av746ePSf2p/IRa5/tPrZc/xPU5FXiCpaakSNSJKJl1eQdKMEfn5Z69DOk92icPf/TjKiUj5DLcmIlLsFmAPvHQg4SXCGTZ/mNehOGrb4W38uOlH7ml9z9ldxpzk5eZN2WnQwF5R7dEV5E7ouCWN6oeSqD/oaeqWr8vAWQO58J0Lee/39ziVcsrr8PwXEwPHj9sd7bKwbNcy2n3cjid+eoLOF3ZmzaA1PNT2Ib+36i1Tsgxjbx3Lxzd9zC/bfqHNh22CYrOtqWum0njuSm5bcATz5JPQsaPXIfkvq2yS+QZ0Bypk+LkicKs/xwbDzakeh4jIoG8GSdjzYUE/4ZYXQ+cOFRNrZMuhLe6cYPBgkfDwoFheeI5LLxW59lqvo8i/u+6yK3ASEyUtLU1+3PSjdBjd4UwdrOELhktiUqLXUeYuKUmkUiWRO+885+4TSSfk8e8fl5ChIVJzWE2ZvGpygRenLN+1XC565yIJHRoqr/36mt+9FqelpqXKNS83lUMRIZJ2ySUip097EkduKOBQ1Yos7lvuz7HBcHMycWw6uElChobIkz8+6VibXkpNS5UGwxvIdZ9e595JOnUSadfOvfbz6+67RWrW9DqK/Dlxwg7v3HffOXenpaXJ/zb/T64ac5UQi9R4vYa8Mf8NOZEUhIUOM7rrLpGKFc+8gf6w8QdpOLyhEIs88NUD564iK6Ajp45Iz0k9hVjkps9ukgOJBxxr219f/vGF/NAISS5dSmTt2oCf31/ZJQ5/rxzP6nkObdJQuDSq1IiY5jF8sOQDjp0+5nU4Bfa/Lf9j6+Gtzl8p7iNBVGoks6goW9ahMBbamzHDDu/073/O3cYYOjXsxNy75zL3rrlEVo/ksR8eo+HbDXn9t9c5nnTco4BzERMDhw9z9NsZ3DX9LjqP70xYaBhz75rLRzd9RMXSFR07VflS5ZnUcxLvdHmH7zZ+R/SH0Szesdix9nOTJmlsjv0H12+GkDffgqZNA3Zux2SVTTLfsBf8vQlciC11/hbwX3+ODYabkz0OEZFFCYuEWOTN+W862q4Xek3uJZVeqSQnk7OfmCyQHTtsx/add9xpvyC++87GNneu15HkXdeudhVOau5DLfO2zZPO4zoLsUjV16rKf+b9R46eOhqAIP2XlpgoSRGlZWy7UlLi+RLyzOxn3PubzGBRwiK54K0LJOz5MHl30bsBuU7rh6nD5FQosv3q6KC/7oYCDlWVAV4BlqTfXgbK+HNsMNycThwiIh3HdJR6b9ZzZy+FADmQeEBKvVBK/vbN39w7yaxZ9s8sCFeMSEKCje2997yOJG927xYJDRV56qk8HTb/z/lnlvFWfrWyvPjzi3L45GGXgvTf1kNbpcv4LvJZFHKgXAmJ37E8oOc/kHhAbvzsRiEWuW3SbXLklHvX9qSeOC7ra5eSveVCJWV3NlUAgkh2icPfVVUnRORJSa82KyJPi0jxKBebjSHth7D96Ha+XP2l16Hk24T4CZxOPc190S4NU8HZFVUtW7p3jvyqXRsqVix8K6s+//zsnht5cHm9y5nVdxa/3/877eu159k5z9Lg7QYMnTuUw6cOuxNrDlLTUhm+cDiRIyKZt20eFfrcQ+VjKbRcH9hYKodXZkavGbx63atMXTOVth+1JW53nCvn2vTgbTTeeZqVrz5GaI2arpwjILLKJplvwI9AxQw/VwK+9+fYYLi50eNITUuV5u81l9YjWxfKMiRpaWly8QcXS/SH0e6eqE8fkXr13D1HQVxxhS0oV5hccolIdMH/3ZbuXCq3TLxFiEXK/6e8PPe/5wI2URy3O04u/ehSIRa5YcINsu3wNpHjx+3qu7+52APOxS9bf5Hab9SW0i+Wlk+WfuLo/+3Ub23v+9OrKgbdhYjZoYCT41VF5HCGZHMIP/YcL8pCTAiPt3+cFbtXMHvLbK/DybNlu5YRtyfOvUlxn7i44Cilnh3fboBSwM2TAmXNGli69LxJ8fyIrhXN9F7TWf7gcq5vdD0v/PICDYY34JnZz7A/0Z0FAyeTT/L07Ke55KNL2Hp4KxNjJjKz90zqV6hv953o0gWmTrV703vgyguuZPmDy+lQvwP3f30/d8+425m9ePbt43T/PqysBiVff8ud66UCKatskvkGLAXqZ/i5AbDMn2OD4eZGj0PE1neqOaymdB7X2ZX23TRw5kAp/WJpR5c5nufUKTsW//TT7p2joN59185z7Cgk1+U8/bR9TXfvdrzp+N3xcvuXt4uJNVLmpTLyxI9PyN7jex1rf86WOdL4ncZCLHL39LuzLqY5frz995g/37Hz5kdKaorEzokVE2sk8v3IXEub5CgtTdJuuVlOlzBy8xMXFJrehkj2PQ5/E0cX4E9gXPptG/BXf44NhptbiUNE5OVfXhZikRW7Vrh2DqclJiVKhf9UkL5T+rp7ouXL7Z/Y55+7e56CmDPHxvj9915HkrvUVLuSqksXV0+zcs9K6T25t5hYIxEvRcjj3z8uu4/lP1EdTDwo9824T4hFGr3dKOeCg4cPi4SFiTz2WL7P56QfN/0o1V6rJmVeKiMT4ifkr5GPPhIB+Wdn8t+GRwqUOOzxVAeexe781xPo6O+xXt/cTBwHEw9KmZfKSP+p/V07h9PGx40XYpHZm2e7e6L//tf+ia1Z4+55CmLvXhvjG294HUnufJVkJwTmzWfNvjXSb2o/CRkaIuEvhsvgbwfLzqM7/T4+LS1NJq2cJDVeryGhQ0PlXz/8y78LEW+4wVb8DZK5wx1Hd8iVo68UYpEHv34wb8uE162TtIgIWdC0jDR/p2mh6m2IFLzHcT/wB3AImAOcpJiVVc/J4G8HS4nnS8ifh/909TxO6fTfTtJweEP3yy388592r4XkIN9Mp3p1kXvv9TqK3N1/v71a/ERgrwJfv3+93D39bgkdGiqlXiglf5/1d0k4kpDjMduPbJebPrtJiEWiP4yWZTvzsCvhqFH2rWnp0gJG7pzk1GR54scnhFik9cjWsvHAxtwPSkoSadtWTlUoK7X/iXwW/5n7gTqsoInjD6A06aVHgGbAF/4cGww3txPH1kNbJXRoqDz2fXB0r3Oy8cBGIRZ54ecX3D/ZtdfaFUDB7pprgrMkSkYnT4pUqHBePadA2nhgo9w34z4p8XwJKflCSRk4c+B5H5ZS01LlvUXvSbmXy0n4i+Ey7Ldhed+Fb//+oJ0b+3rd11LplUpS/j/lZcrqKTk/+emnRUAGP1BPmr3XrND1NkQKnjgWp39dAZTyfe/PscFwcztxiIj0ntxbyr1cLiguqMrJM7OfkZChIbL9yHZ3T5SWJlK1auH4JP/IIyJlyvh1FbZnJk2y/11/9H5Dos0HN8uArwZI2PNhEvZ8mDz49YOy9dBWWbV3lbQf1f5M+fJc90nPyXXXiTRtGjTDVRltPbRV2n3cTohFBn87WE6nZFGg8JdfRIyRzd07CbGFs7chkn3i8Hc5boIxpiIwHfjRGDMD2Onvyq3iYEj7IRxLOsaHSz/0OpRspaal8t8V/+WvF/6VuuXrunuyPXtsDahgrFGVWVQUnDgB27Z5HUn2xo+3Fyx26uR1JDSs1JAPb/qQjY9s5P7o+xmzYgwXvXsRrUe2Zu3+tYy9dSzf9/ueRpUa5f8kMTGwbh2sXu1c4A65oOIFzLtnHo+0e4Thi4bTcUxH/jzy59knHDgA/fsjjRrR68rdNKvajNsjb/cuYDdklU1yugFXATcDJfN6rFe3QPQ4RESuHXut1H6jdtafQILArPWzhFhk8qrJ7p/MVwdqzhz3z1VQ8+fbWL/6yutIsrZvn91T/PHHvY4kS9uPbJfB3w6WQd8Mcm6TpF27RIwRGTrUmfZc8uWqL6Xcy+Wk8quV5Zv139h5jU6dREqVkp8mvlyoexsiBe9xZEw0P4vIVyKS5HAOK/SGtB/CzmM7mfjHRK9DydKo5aOoFlGNm5re5P7JgrnUSGYtWtivq1Z5G0d2vvgCUlIcuejPDXXL1+WtLm/x3g3vUb2MQ9cF16wJHTrAlODeL65ni54se3AZ9crXo9tn3Vhw22UwZw5pH45k8KHPimZvg6zLpat86nxhZ1pWb8mwBcN8vbOgse/EPr5a9xX9W/WnZGhJ908YHw916kCVKu6fq6AqVIB69YK3ZtX48XbIrzAM+zkpJsb+HW3c6HUkObqo8kUsuG8B4/Z24PIZy5jYpS7vNzvGyr0r+b+O/1f4rxLPgiYOBxljeLz946zcu5LvNn7ndTjnGBc/juS0ZO5tc29gThjspUYy85UeCTYbNsDChUHb23BVjx72a5D3OgDCFyym30cL2XFFKwZccYBHvnukyPY2QBOH43pF9aJOuTq8Pv91r0M5Q0QYtXwUf6nzFyKrR7p/wqQkW1OpMH1CjoqyMaekeB3JucaPB2Ogd2+vIwm8evWgXTuYPNnrSHK2bZvtHTVqRJ2ZP7PowSV0vrAzb3d5u0j2NkATR84WL4aZM2HHDr+L4JUMLcngywYzZ+sclu5c6nKA/lm0YxGr9612v6Chz9q19g24sCWOpKTgGhYRsYnj2mvtsF9xFBMDS5YE74q348fh5pshORm++goqVqRFtRZ83+97Ol/Y2evoXKOJIycffAA33QR160KNGrZy51NPwaRJ9g0mmwqeAy4ZQPlS5Rm2YFiAA87aqGWjiAiL4I6oOwJzwrj0vQwK01BVZHpPLJgmyBcsgM2bi+cwlU9MjP06daq3cWQlLQ3uvtsOcX7xReHcAjafNHHk5O23Yd48eOcduPFGuz/1sGFwxx3QuLHdBKhjRxg8GMaOtRN5ycmUL1WeAdED+HLVl2w9vNXTX+FE0gk+X/U5t0feTvlS5QNz0vh4KFkSmjQJzPmc0Ly5HRIKpnmOceMgPBy6d/c6Eu9ceKH9ABKM8xwvvGDjev11+OtfvY4moEp4HUBQK1fOLgns0OHsfadP20+ly5bB8uX29vHHkJhoHy9VClq25N+RTUg8KEwd8wT/fHAMRER48it8ufpLjicdD9wwFdjEERkJJQrRn1dEhH2TCpbEkZRkP8V2727/DouzmBj4979h1y6oVcvraKypUyE2Fu68Ex591OtoAs4E27JRN7Rt21aWLFni3glSU2H9eptEMiaUQ4cAkJAQTLNmEB0NbdqcvVWs6F5M6a4ccyV7T+xl7aC1GGNcPx9g1+B37QpjxgTmfE7p3t3Oz6xZ43UkMH26jefbb+0QaXG2erX9IPL++zBwoNfR2KHY9u3tNUpz50Lp0l5H5BpjzFIRaZv5/kL0kTCIhYbaoY7mzaFPH3ufCGuXfM9Tb3TlsdJX0WF/BPzvf3ay06dhQ5tAMiYUBz9Rrdu/jl///JVXrn0lcEljzx57K0wT4z5RUfD117ZXWaqUt7GMG2fn1a67zts4gkGLFtCsmR0W8jpx7NsHt9xiP/RNm1akk0ZONHG4xRiaXdqFkzf+lZ67V7B18FZKlyht31R9PRLfLePEX82aZ5OIL6E0bGjH3/No9PLRhJpQ7mp9l4O/WC7++MN+LYyJIzLS9h7XrfM2/kOH7Gq+gQML13Cfm2Ji4JVXbP2zqlW9iSEpCXr2tP+Hf/kleIbNPODq5LgxposxZp0xZqMx5sksHjfGmHfSH483xkRneOwfxpiVxphVxpjBGe5vbYxZaIxZYYxZYoxp5+bvUFBD2g9hz4k9jI9P72lkXp21YQMcPgw//wzDh0PnzpCQAK++av9IL7wQKlWyY7x5kJyazNi4sXRr0o2aZWs6/ntly7eiqjAmjqgo+9XreY4vv7RvUsV5NVVmMTE2qc+Y4V0MjzxiE8aoUXDppd7FEQyyKmDlxA0IBTYBjYCSQBzQItNzbgC+BQxwGbAo/f4oYCUQge0V/QQ0Tn/sB6BrhuPn5hZLoIocZiUtLU3ajGwjTd9tmreNkxITRX7/XeTDD+2OaCDy009+Hz59zXQhFpmxdkY+oi6AO+8UqVUrsOd0yunTtpjgU095G0eHDiLNmwdlSXHPpKWJNGwo0rWrN+cfMcL+H3ziCW/O7xGcKnKYB+2AjSKyWWxBxM+BWzI95xbg0/QYFwIVjTG1gObAQhFJFJEU4GfAtyZRAN+60goEeXl3YwxD2g9h3YF1zFw/0/8Dw8Ptp5oBA+yVsxddBA88YMt/+2HU8lHULFuTGxrfkM/I8yk+vnD2NsAuIW7a1Nsex5Yt8OuvtrcRqHmpwsAY2+v46SfbQw+kOXNsb6NbN3jppcCeO0i5mTjqANsz/JyQfp8/z1kJdDTGVDHGRGB7FvXSnzMYeN0Ysx0YBjyV1cmNMQPSh7KW7Nu3r6C/S4HcFnkb9SvUZ9j8fF4QGB4On3xi31Seey7Xp+86totZG2Zx18V3USIkgGPkycl2BUxhuvAvM69rVk2YYL/27etdDMEqJsb+jc3Mwwewgtq8GW67zV639dlndiGMcjVxZPVxKfPa3yyfIyJrgFeBH4HvsMNcviJCDwOPikg94FFgVFYnF5GPRKStiLStVq1afuJ3TImQEjx62aPM+3MeixIW5a+Rq66Chx6y8yALF+b41E/jPiVVUgNX0NBn3To7Nl9YexxgJ8i3bPG7Z+coEbua6qqroH79wJ8/2LVrZ0uvBOpiwGPH7Aoq39xK+QBdQFsIuJk4EjjbSwCoy/nDStk+R0RGiUi0iHQEDgIb0p9zF+BbhvQldkgs6N0ffT8VS1csWPHDV1+1/3Huu88uGc2CiDB6xWg61O9AkyoBvnLbtwdHYU4cvglyL3aeW7zYXg+kk+JZCwmxFXO/+87WiHJTWpq9uG/1aruIpXFjd89XyLiZOBYDjY0xDY0xJYFewFeZnvMVcGf66qrLgCMisgvAGFM9/Wt9oAfg2x1pJ3YXQoBrOJtQglrZkmV5uO3DTF0zlY0H81lIr3x5GDnS/jH/5z9ZPuXXP39l/YH1gb1S3CcuDsLC7Jr7wsrLlVXjx9vrR3r2DPy5C4uYGDh1yl4Y6abYWHsR5ptvwvXXu3uuwiirGXOnbti5ifXY1VXPpN/3EPBQ+vcGeD/98T+AthmOnQesxg5TXZvh/g7A0vT7FwGX5BaHl6uqMtp5dKeUfKGkPDzz4YI11LevSFiYSHz8eQ/dNe0uKfdyOTl++njBzpEfXbqIXHxx4M/rpJQUkdKlRf75z8CeNylJpFo1kdtuC+x5C5uUFJHq1UXuuMO9c3zxhV1Bde+9xX5lG9msqvJ8P/BA3IIlcYiI3DfjPin9YmnZe3xv/hvZt0+kalWRSy+1/5HSHTl1RCJeipAHvnrAgUjzoXZtkf79vTm3k6KjRTp3Duw5v/5agnrf82AyYIBI2bIiJ0863/ayZSLh4SLt24ucOuV8+4VMdolDq+MG2GOXP8aplFOMWDwi/41UrQrvvmvHxN9++8zdn6/8nMTkRG+Gqfbvh507C/eKKp/IyMCXVx8/3v67Fve6VP6IibFzHD/84Gy7e/bYyfAqVWw1B6/LzgQxTRwB1rxac25sciPvLX6PxOTE/Dd0xx12r5Bnn4VNmwBbYiSyWiTt6niwXqAoTIz7REXZzbvSi1S67sgRu2qnVy87R6Ry1qmTrabg5OqqpCSbkPbvt/8WNWo413YRpInDA0PaD2F/4n7Grhib/0aMgREj7BvNAw+was9KFu1YxH1t7gtcQcOMilrigMD1OqZMsRO+/foF5nyFXViY3XXvq6/sG35Bidi6YL/9Zis6R0fnfkwxp4nDA1fWv5J2ddrxxoI3SE1LzX9DdevaTWTmzGHlfx4lLCSMfq08evOJi7Of0orCJ7VAr6waP94u92xXKFaWB4eYGHsF+Zw5BW/rvfds/alnnrE9eZUrTRweMMbw+OWPs+nQJmasK2DRtvvvJ+2qq+j64WzurnY91cp4dLFjYS41klm9enbzpEAkju3b7Z4OWmIkb66/3v4bFXS4avZsuxHTzTfD8887E1sxoInDIz2a96BRpUa8Pv913zLj/AkJ4fsnYghLEV6YfNB2uwMtJcUO6xSVxGFM4CbIJ0yw/2ZaYiRvSpe22zlPn26v7M6PTZtsOZFmzewV+yH6dugvfaU8EhoSyj8v+ycLExby2/bfCtTWOwdn8UbXCtSYvdCW5A609evtlexFYUWVT1SU3VvEzUTsKzFyxRXQqJF75ymqYmLsxkrz5uX92KNHbS/DGC0nkg+aODx0T5t7qBJepUBlSLYf2c73G78n+e+DoG1b+Nvf4MABB6P0Q1GaGPeJirKv49697p1jxQpbBUBLjORPly62AGheh6tSU20Pb906+0Hrwgvdia8I08ThoYiwCAZdOoiv1n3F2v1r89XG2LixCMJdbe+zE3yHDtkx20CKj7c71RXmUiOZBWKCfNw4W8r9ttvcO0dRVqaM3dt+6lRbW8pfzz1nK+wOHw7XXONaeEWZJg6PDWo3iNIlSvPmgjfzfGyapDF6+WiuaXgNjSo1sp/4n3rKviG5Xcsno7g4u996Ubpgyu3EkZICEyfaPR4qV3bnHMVBTIy98HSRn1WnJ060dd4eeAAGDXI3tiJME4fHqpepzl0X38WncZ+y5/iePB07d+tcthzecu6V4s88Y9/EH3zQloUOhKK0osqnenV7BbFbE+SzZ8Pu3TpMVVA33mh7bf4MVy1dCvfeCx062CW4uoot3zRxBIHHLn+MpNQk3v393TwdN2r5KCqUqkD3Zt3P3lmqlB2ySkiwvQ+3HTxoz1XUEocx7m7qNG6cvfr5hgDv0FjUlC9vl+ZOmZLzQobdu205kerV7XNLlgxcjEWQJo4g0LhKY25tdisjFo/geJJ/+wwcOnmIKaun0LdlX8LDws998PLL7VaX77+fvxUneeGbGC9KK6p8fInD6ZVVx4/DtGlw++1Fa3jPKzExsHUrLF+e9eOnT0P37nb+b8YMmzxUgWjiCBJD2g/h0KlDjF4+2q/nf/bHZ5xOPc190dkUNHzxRWjQAO6/H06edC7QzIriiiqfqCg73Ld9e+7PzYtp0yAxUYepnHLzzXZL16yGq0TszpkLF8LYsdC6dcDDK4o0cQSJy+tdTvt67Xlr4VukpKXk+vxRy0fRumZromtlU1enbFn4+GN7jYWbV8TGxUG1alCzpnvn8IpbE+TjxkHDhtC+vbPtFldVqtjCh5Mnn987fPtt+O9/4f/+TzfIcpAmjiAypP0Qth7eypTVOU/0Ld+1nOW7l+dePv266+xk4Ouvw7JlDkaagW9ivChONEZG2q9OTpDv3Gknxvv1K5qvmVdiYuyHpIz/Vj/8AI89Zoep/v1v72IrgjRxBJGbm95MkypNci1DMnr5aEqFlqJvSz/KVAwbZnsE990HyckORou9kGrlyqI5TAV28rp2bWd7HBMn2msOdJjKWbfeahOxb7hq/XpbsDAyEj79VMuJOExfzSASYkJ47PLHWLprKXO3zs3yOadSTjHhjwn0aN6DSuGVcm+0UiVbfn3FCptEnLRhgy0HXhQnxn2cXlk1bhz85S+2Gq5yTs2adpntlCl2fxPfvMeMGXbYVjlKE0eQ6d+qP9UiqjFsQdZv8tPWTOPQqUPc2+Ze/xvt3t2O7w4dCmvzd4V6loryxLhPVJQtC5LfQnoZ/fGHnRPSfTfcERNjX+MuXWwBw8mT7VyScpwmjiATHhbO39v9nVkbZrFq7/lj66OWj6JBxQZc0zCPpRLefRciIuwqq7yUZ8hJfLz9VNe8uTPtBaOoKNur2ry54G2NH29Ls/TqVfC21Pl69LBfFy60f+9XX+1pOEWZJo4gNPDSgUSERZzX69hyaAuzt8zmntb3EGLy+E9Xsya89Zbd5eyDD5wJNC7O1qcqXdqZ9oKRUxPkqam2hHrXrnZvceW8evXsB6Nnn7VLcJVrNHEEoSoRVbi39b1MiJ/AjqM7ztw/ZsUYDIa7W9+dv4bvvBM6d4Ynn4Rt2woeaFEsNZJZixb2a0HnOebOtfuY6zCVuz7+GF54wesoijxNHEHq0csfJVVSeWfROwCkpqUyZsUYOl/YmfoV6uevUWPgww/PXhRVkCuiDx+GP/8s+omjbFk7Tl7QxDF+vC2PcdNNzsSllIc0cQSpRpUaEdM8hpFLR3L09FF+2vwTCUcTcr92IzcNGtjqoN99Z9/M8qsolxrJrKArqxIT7UTtbbfZ/SOUKuQ0cQSxIe2HcPT0UT5Z9gmjlo+iSngVbm56c8EbHjjQXrU8eDDsyVtF3jOKw4oqn6gou+lPUlL+jp8xw9an0mEqVURo4ghil9a5lKsuuIph84cxfe10+rXqR6kSDhTFCw2FTz6xb2aPPJK/NuLj7T4StWsXPJ5gFxlp98/YsCF/x48fD/XrQ8eOzsallEc0cQS5Ie2HsOv4LpLTkgs+TJVR8+a2fs+kSTB9et6Pj4uzw1TFoWxGQWpW7dkD339vtyrVq5dVEaF/yUGua+OutKzeksvrXk7LGi2dbfxf/7JDTQMH2slufxX1UiOZNW1qe2n5SRyff25fLx2mUkWIJo4gF2JCmHPXHGb2mel842FhMHq0/VQ8ZIj/x23ebCd8i0viKF3algjJT+IYPx6io88u61WqCNDEUQhUiahC5XCX9qW+5BJ4/HE75zF7tn/HxMXZr8VhRZVPflZWrV0LS5ZoQUNV5GjiUBAbaz9RDxgAJ07k/vz4eDteX5w+RUdG2vpHedkUa9w4+zppiRFVxLiaOIwxXYwx64wxG40xT2bxuDHGvJP+eLwxJjrDY/8wxqw0xqwyxgzOdNzf09tdZYx5zc3foVgID7c9js2b7YR5buLjoUmT4nVNQlSUvWByzRr/np+WZoepOncumptcqWLNtcRhjAkF3ge6Ai2A3saYzB9RuwKN028DgA/Sj40CHgDaARcDNxpjGqc/1gm4BWglIpGAw7XCi6mOHeHhh2H4cFi0KOfn+lZUFSd5XVn166/2ynodplJFkJs9jnbARhHZLCJJwOfYN/yMbgE+FWshUNEYUwtoDiwUkUQRSQF+BrqnH/Mw8IqInAYQkb0u/g7Fyyuv2Osy7rsv+4vdjhyBrVuLz8S4z0UXQcmS/ieOceOgTBm4JfOfvFKFn5uJow6wPcPPCen3+fOclUBHY0wVY0wEcANQL/05TYArjTGLjDE/G2MuzerkxpgBxpglxpgl+/btc+DXKQbKl4eRI20l2P/8J+vn+N44i1viKFHCXvviT5XcU6fgyy/t/hBlyrgfm1IB5mbiyOrKsMxV9bJ8joisAV4FfgS+A+KAlPTHSwCVgMuAIcAkY86/Ck1EPhKRtiLStlq1avn8FYqhbt3sxWovvZT1p+viuKLKJzLSvx7HzJm2Z6bDVKqIcjNxJHC2lwBQF9jp73NEZJSIRItIR+AgsCHDMVPTh7d+B9IA3eDAScOHQ4UKcO+95+98Fx8PFStC3bpeROatqCg7b3H0aM7PGzfODvl16hSYuJQKMDcTx2KgsTGmoTGmJNAL+CrTc74C7kxfXXUZcEREdgEYY6qnf60P9AAmph8zHbgm/bEmQElgv4u/R/FTtardQW3xYnj77XMf8+3BURxKjWTmmyDPabhq/36YNQv69LFXmytVBLmWONIntf8GfA+sASaJyCpjzEPGGN/2XLOAzcBG4GNgYIYmphhjVgNfA4NE5FD6/aOBRsaYldgJ97tECrKxhMrSHXfYvSOefdZevwB2iWl8fPEcpgL/VlZNmmQLIuowlSrCSrjZuIjMwiaHjPeNzPC9AIOyOfbKbO5PArTwj9uMgREj7Lj+Aw/Yq8q3bLEXCBa3iXGfCy6wk9059TjGjbOvT3F9jVSxoFeOq+zVrQuvvw5z5sCoUcVrD46s+K6Wz67HsWEDLFyoBQ1VkaeJQ+Xs/vvh6qvhscfg229tT8Q3ZFMc5VSzasIE+/r06RPYmJQKME0cKmchIfDxx/aCwI8/tjWtIiK8jso7UVG2mnDma4NEbImRa6+FOpkvV1KqaNHEoXJ30UXwwgv2++I6TOWT3cqqBQvsIgIdplLFgKuT46oIGTzYvln26OF1JN7KmDiuvvrs/ePH26KPxf31UcWCJg7lnxIlYMwYr6PwXq1a9gLIjPMcSUnwxRfQvTuUK+dZaEoFig5VKZUXvsUBGRPHrFlw8KAOU6liQxOHUnnlSxy+607Hj4caNeD6672NS6kA0cShVF5FRcHhw7BzJxw6BF9/Db172+E8pYoB/UtXKq8yTpBv3WrnOHSYShUjmjiUyqvISPt15UqYNs3u0xEdnfMxShUhOlSlVF5VrWrnNL7+2m4R279/8awWrIotTRxK5UdUFMyda7/v29fTUJQKNE0cSuWHb57jqqugfn1vY1EqwDRxKJUfvsSh+26oYkgnx5XKj1tvhdWroVcvryNRKuA0cSiVH1Wrwptveh2FUp7QoSqllFJ5oolDKaVUnmjiUEoplSeaOJRSSuWJJg6llFJ5oolDKaVUnmjiUEoplSeaOJRSSuWJEd8uZkWYMWYfsC2fh1cF9jsYTmGnr8dZ+lqcS1+PcxWF1+MCEamW+c5ikTgKwhizRETaeh1HsNDX4yx9Lc6lr8e5ivLroUNVSiml8kQTh1JKqTzRxJG7j7wOIMjo63GWvhbn0tfjXEX29dA5DqWUUnmiPQ6llFJ5oolDKaVUnmjiyIExposxZp0xZqMx5kmv4/GKMaaeMWaOMWaNMWaVMeYfXscUDIwxocaY5caYmV7H4jVjTEVjzGRjzNr0v5PLvY7JK8aYR9P/n6w0xkw0xpT2OianaeLIhjEmFHgf6Aq0AHobY1p4G5VnUoDHRKQ5cBkwqBi/Fhn9A1jjdRBB4m3gOxFpBlxMMX1djDF1gEeAtiISBYQCRW5/YU0c2WsHbBSRzSKSBHwO3OJxTJ4QkV0isiz9+2PYN4U63kblLWNMXaAb8InXsXjNGFMe6AiMAhCRJBE57GlQ3ioBhBtjSgARwE6P43GcJo7s1QG2Z/g5gWL+ZglgjGkAtAEWeRyK14YD/wLSPI4jGDQC9gFj0ofuPjHGlPE6KC+IyA5gGPAnsAs4IiI/eBuV8zRxZM9kcV+xXrtsjCkLTAEGi8hRr+PxijHmRmCviCz1OpYgUQKIBj4QkTbACaBYzgkaYyphRyYaArWBMsaYft5G5TxNHNlLAOpl+LkuRbDL6S9jTBg2aUwQkalex+OxK4CbjTFbsUOY1xhjxnsbkqcSgAQR8fVCJ2MTSXF0HbBFRPaJSDIwFWjvcUyO08SRvcVAY2NMQ2NMSewE11cex+QJY4zBjl+vEZE3vY7HayLylIjUFZEG2L+L/4lIkftU6S8R2Q1sN8Y0Tb/rWmC1hyF56U/gMmNMRPr/m2spggsFSngdQLASkRRjzN+A77ErI0aLyCqPw/LKFUB/4A9jzIr0+54WkVnehaSCzN+BCekfsjYD93gcjydEZJExZjKwDLsacTlFsPSIlhxRSimVJzpUpZRSKk80cSillMoTTRxKKaXyRBOHUkqpPNHEoZRSKk80cSgV5IwxV2sFXhVMNHEopZTKE00cSjnEGNPPGPO7MWaFMebD9P06jhtj3jDGLDPGzDbGVEt/bmtjzEJjTLwxZlp6jSOMMRcZY34yxsSlH3NhevNlM+x3MSH9qmSlPKGJQykHGGOaA3cAV4hIayAV6AuUAZaJSDTwM/Dv9EM+BZ4QkVbAHxnunwC8LyIXY2sc7Uq/vw0wGLs3TCPs1fxKeUJLjijljGuBS4DF6Z2BcGAvtuz6F+nPGQ9MNcZUACqKyM/p948FvjTGlAPqiMg0ABE5BZDe3u8ikpD+8wqgAfCr67+VUlnQxKGUMwwwVkSeOudOY57L9LycavzkNPx0OsP3qej/XeUhHapSyhmzgZ7GmOoAxpjKxpgLsP/HeqY/pw/wq4gcAQ4ZY65Mv78/8HP6HicJxphb09soZYyJCOQvoZQ/9FOLUg4QkdXGmGeBH4wxIUAyMAi7qVGkMWYpcAQ7DwJwFzAyPTFkrCbbH/jQGPN8ehu3BfDXUMovWh1XKRcZY46LSFmv41DKSTpUpZRSKk+0x6GUUipPtMehlFIqTzRxKKWUyhNNHEoppfJEE4dSSqk80cShlFIqT/4fTV3lfRkayCEAAAAASUVORK5CYII=",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "# todo : set suitable learning rate (1, 0.1, 0.01, 0.001)\n",
        "lr = 0.001 #\n",
        "model = Model(SGD(learning_rate=lr))\n",
        "#################################################################################\n",
        "# todo : add the layers to the model                                            #\n",
        "#        choose suitable l2_coef for both linear layers (1e-2, 1e-3, 1e-4, 1e-5)#\n",
        "# ###############################################################################\n",
        "# model.add(Linear(\"test\", 100, 100, l2_coef=1e-2))\n",
        "# model.add(ReLU(\"test\"))\n",
        "# model.add(Linear(\"test\", 100, 10, l2_coef=1e-2))\n",
        "# model.add(SoftmaxCrossentropy(\"test\"))\n",
        "my_linear = Linear(\"test\", 3072, 100, l2_coef=1e-4)\n",
        "model.add(my_linear)\n",
        "my_relu = Sigmoid(\"test\")\n",
        "model.add(my_relu)\n",
        "my_linear2 = Linear(\"test\", 100, 10, l2_coef=1e-4)\n",
        "model.add(my_linear2)\n",
        "softmaxcrossentropy = SoftmaxCrossentropy(\"test\")\n",
        "model.add(softmaxcrossentropy)\n",
        "#################################################################################\n",
        "#                             end of code                                       #\n",
        "#################################################################################\n",
        "history = model.fit(X_train_flattened, y_train_squeezed, X_val_flattened, y_val_squeezed, batch_size=1024, epochs=10)\n",
        "plot_history(history)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "Copy of Part1.ipynb",
      "provenance": [],
      "toc_visible": true
    },
    "interpreter": {
      "hash": "f25003f7e2a58fe7a2136f4a1806bc5c1cb2dd7a1c980b3cbe05b6c5f6f01de6"
    },
    "kernelspec": {
      "display_name": "Python 3.10.4 ('ML_For_Bio_miniforge')",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.4"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}