week02_classification/homework_part1.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Homework part I: Prohibited Comment Classification (3 points)\n",
    "\n",
    "![img](https://github.com/yandexdataschool/nlp_course/raw/master/resources/banhammer.jpg)\n",
    "\n",
    "__In this notebook__ you will build an algorithm that classifies social media comments into normal or toxic.\n",
    "Like in many real-world cases, you only have a small (10^3) dataset of hand-labeled examples to work with. We'll tackle this problem using both classical nlp methods and embedding-based approach."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>should_ban</th>\n",
       "      <th>comment_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>50</th>\n",
       "      <td>0</td>\n",
       "      <td>\"Those who're in advantageous positions are th...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>250</th>\n",
       "      <td>1</td>\n",
       "      <td>Fartsalot56 says f**k you motherclucker!!</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>450</th>\n",
       "      <td>1</td>\n",
       "      <td>Are you a fool? \\n\\nI am sorry, but you seem t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>650</th>\n",
       "      <td>1</td>\n",
       "      <td>I AM NOT A VANDAL!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>850</th>\n",
       "      <td>0</td>\n",
       "      <td>Citing sources\\n\\nCheck out the Wikipedia:Citi...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     should_ban                                       comment_text\n",
       "50            0  \"Those who're in advantageous positions are th...\n",
       "250           1          Fartsalot56 says f**k you motherclucker!!\n",
       "450           1  Are you a fool? \\n\\nI am sorry, but you seem t...\n",
       "650           1    I AM NOT A VANDAL!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n",
       "850           0  Citing sources\\n\\nCheck out the Wikipedia:Citi..."
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "data = pd.read_csv(\"comments.tsv\", sep='\\t')\n",
    "\n",
    "texts = data['comment_text'].values\n",
    "target = data['should_ban'].values\n",
    "data[50::200]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "texts_train, texts_test, y_train, y_test = train_test_split(texts, target, test_size=0.5, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Note:__ it is generally a good idea to split data into train/test before anything is done to them.\n",
    "\n",
    "It guards you against possible data leakage in the preprocessing stage. For example, should you decide to select words present in obscene tweets as features, you should only count those words over the training set. Otherwise your algoritm can cheat evaluation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preprocessing and tokenization\n",
    "\n",
    "Comments contain raw text with punctuation, upper/lowercase letters and even newline symbols.\n",
    "\n",
    "To simplify all further steps, we'll split text into space-separated tokens using one of nltk tokenizers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "before: How to be a grown-up at work: replace \"fuck you\" with \"Ok, great!\".\n",
      "after: how to be a grown-up at work : replace \" fuck you \" with \" ok , great ! \" .\n"
     ]
    }
   ],
   "source": [
    "from nltk.tokenize import TweetTokenizer\n",
    "tokenizer = TweetTokenizer()\n",
    "preprocess = lambda text: ' '.join(tokenizer.tokenize(text.lower()))\n",
    "\n",
    "text = 'How to be a grown-up at work: replace \"fuck you\" with \"Ok, great!\".'\n",
    "print(\"before:\", text,)\n",
    "print(\"after:\", preprocess(text),)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# task: preprocess each comment in train and test\n",
    "\n",
    "texts_train = <YOUR CODE>\n",
    "texts_test = <YOUR CODE>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "assert texts_train[5] ==  'who cares anymore . they attack with impunity .'\n",
    "assert texts_test[89] == 'hey todds ! quick q ? why are you so gay'\n",
    "assert len(texts_test) == len(y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Solving it: bag of words\n",
    "\n",
    "![img](http://www.novuslight.com/uploads/n/BagofWords.jpg)\n",
    "\n",
    "One traditional approach to such problem is to use bag of words features:\n",
    "1. build a vocabulary of frequent words (use train data only)\n",
    "2. for each training sample, count the number of times a word occurs in it (for each word in vocabulary).\n",
    "3. consider this count a feature for some classifier\n",
    "\n",
    "__Note:__ in practice, you can compute such features using sklearn. Please don't do that in the current assignment, though.\n",
    "* `from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "example features: ['!', '12:20', '300', '_', 'adorned', 'alternative', 'archive', 'average', 'benkner', 'bout', 'came', 'chest', 'combined', 'consumers', 'cricket', 'decisions', 'dickheads', 'domestic', 'eductaion', 'essentially', 'faggot', 'firms', 'frustrated', 'goal', 'hanibal', 'hip-hop', 'identified', 'infoboxes', 'issue', 'kindergarten', 'lets', 'lot', \"mclaren's\", 'moderator', 'naturally', 'noticeable', 'opposing', 'pdf', 'plant', 'pretoria', 'punctuation', 'rebels', 'repetative', 'riadh', 'schulz', 'shes', 'slit', 'spoof', 'stupid', 't', 'theoretical', 'topic', 'uglyness', 'userspace', 'wanted', 'wikieditor', 'year', '←']\n"
     ]
    }
   ],
   "source": [
    "# task: find up to k most frequent tokens in texts_train,\n",
    "# sort them by number of occurences (highest first)\n",
    "k = 10000\n",
    "\n",
    "<YOUR CODE>\n",
    "\n",
    "bow_vocabulary = <YOUR CODE>\n",
    "\n",
    "print('example features:', sorted(bow_vocabulary)[::100])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def text_to_bow(text):\n",
    "    \"\"\" convert text string to an array of token counts. Use bow_vocabulary. \"\"\"\n",
    "    <YOUR CODE>\n",
    "    \n",
    "    return np.array(<...>, 'float32')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_bow = np.stack(list(map(text_to_bow, texts_train)))\n",
    "X_test_bow = np.stack(list(map(text_to_bow, texts_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "k_max = len(set(' '.join(texts_train).split()))\n",
    "assert X_train_bow.shape == (len(texts_train), min(k, k_max))\n",
    "assert X_test_bow.shape == (len(texts_test), min(k, k_max))\n",
    "assert np.all(X_train_bow[5:10].sum(-1) == np.array([len(s.split()) for s in  texts_train[5:10]]))\n",
    "assert len(bow_vocabulary) <= min(k, k_max)\n",
    "assert X_train_bow[6, bow_vocabulary.index('.')] == texts_train[6].split().count('.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Naive bayes:__ perhaps the simplest model that can solve your problem is the so called Naive Bayes Classifier. \n",
    "Its a trivial linear model that assumes the independence of input features and computes the coefficients by, well, counting probabilities.\n",
    "\n",
    "If you don't remember the math behind Naive Bayes, read [this chunk](https://lena-voita.github.io/nlp_course/text_classification.html#naive_bayes) to help refresh your memory. Done? Good! Now let's implement that :)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class BinaryNaiveBayes:\n",
    "    delta = 1.0  # add this to all word counts to smoothe probabilities\n",
    "    \n",
    "    def fit(self, X, y):\n",
    "        \"\"\"\n",
    "        Fit a NaiveBayes classifier for two classes\n",
    "        :param X: [batch_size, vocab_size] of bag-of-words features\n",
    "        :param y: [batch_size] of binary targets {0, 1}\n",
    "        \"\"\"\n",
    "        # first, compute marginal probabilities of every class, p(y=k) for k = 0,1\n",
    "        self.p_y = np.array(<YOUR CODE: probability of y=0 and of y=1 in that order>)\n",
    "        \n",
    "        # count occurences of each word in texts with label 1 and label 0 separately\n",
    "        word_counts_positive = <YOUR CODE HERE>\n",
    "        word_counts_negative = <YOUR CODE HERE>\n",
    "        # ^-- both must be vectors of shape [vocab_size].\n",
    "        \n",
    "        # finally, lets use those counts to estimate p(x | y = k) for k = 0, 1\n",
    "        \n",
    "        <YOUR CODE HERE>\n",
    "        self.p_x_given_positive = <...>\n",
    "        self.p_x_given_negative = <...>\n",
    "        # both must be of shape [vocab_size]; and don't forget to add self.delta!\n",
    "        \n",
    "        return self\n",
    "    \n",
    "    def predict_scores(self, X):\n",
    "        \"\"\"\n",
    "        :param X: [batch_size, vocab_size] of bag-of-words features\n",
    "        :returns: a matrix of scores [batch_size, k] of scores for k-th class\n",
    "        \"\"\"\n",
    "        # compute scores for positive and negative classes separately.\n",
    "        # these scores should be proportional to log-probabilities of the respective target {0, 1}\n",
    "        # note: if you apply logarithm to p_x_given_*, the total log-probability can be written\n",
    "        # as a dot-product with X\n",
    "        score_negative = <YOUR CODE HERE - compute unnormalized negative log-probability>\n",
    "        score_positive = <YOUR CODE HERE - compute unnormalized positive log-probability>\n",
    "        \n",
    "        # you can compute total p(x | y=k) with a dot product\n",
    "        return np.stack([score_negative, score_positive], axis=-1)\n",
    "    \n",
    "    def predict(self, X):\n",
    "        return self.predict_scores(X).argmax(axis=-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "naive_model = BinaryNaiveBayes().fit(X_train_bow, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "assert naive_model.p_y.shape == (2,) and naive_model.p_y.sum() == 1 and naive_model.p_y[0] > naive_model.p_y[1]\n",
    "assert naive_model.p_x_given_positive.shape == naive_model.p_x_given_negative.shape == X_train_bow.shape[1:]\n",
    "assert np.allclose(naive_model.p_x_given_positive.sum(), 1.0)\n",
    "assert np.allclose(naive_model.p_x_given_negative.sum(), 1.0)\n",
    "assert naive_model.p_x_given_negative.min() > 0, \"did you forget to add delta?\"\n",
    "\n",
    "f_index = bow_vocabulary.index('fuck')  # offensive tweets should contain more of this\n",
    "assert naive_model.p_x_given_positive[f_index] > naive_model.p_x_given_negative[f_index]\n",
    "\n",
    "g_index = bow_vocabulary.index('good')  # offensive tweets should contain less of this\n",
    "assert naive_model.p_x_given_positive[g_index] < naive_model.p_x_given_negative[g_index]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import roc_auc_score, roc_curve\n",
    "\n",
    "for name, X, y, model in [\n",
    "    ('train', X_train_bow, y_train, naive_model),\n",
    "    ('test ', X_test_bow, y_test, naive_model)\n",
    "]:\n",
    "    proba = model.predict_scores(X)[:, 1] - model.predict_scores(X)[:, 0]\n",
    "    auc = roc_auc_score(y, proba)\n",
    "    plt.plot(*roc_curve(y, proba)[:2], label='%s AUC=%.4f' % (name, auc))\n",
    "\n",
    "plt.plot([0, 1], [0, 1], '--', color='black',)\n",
    "plt.legend(fontsize='large')\n",
    "plt.grid()\n",
    "\n",
    "test_accuracy = np.mean(naive_model.predict(X_test_bow) == y_test)\n",
    "print(f\"Model accuracy: {test_accuracy:.3f}\")\n",
    "assert test_accuracy > 0.75, \"Accuracy too low. There's likely a mistake in the code.\"\n",
    "print(\"Well done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay, it definitely learned *something*. Now let's figure out what exactly it learned. The simplest way to do that is by highlighting which words have a greatest ratio of positive to negative probability or vice versa. We'll go with the positive one [because reasons](https://www.urbandictionary.com/define.php?term=because%20reasons).\n",
    "\n",
    "__Your task__ is to compute top-25 words that have the __highest__ ratio of ${p(x_i | y=1)} \\over {p(x_i | y=0)}$. Enjoy!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# hint: use naive_model.p_*\n",
    "probability_ratio = <YOUR CODE: compute the ratio as defined above, must be a vector of [vocab_size]>\n",
    "top_negative_words = <YOUR CODE: find 25 words with highest probability_ratio, return list of str>\n",
    "\n",
    "assert len(top_negative_words) == 25 and [isinstance(w, str) for w in top_negative_words]\n",
    "assert 'j.delanoy' in top_negative_words and 'college' in top_negative_words\n",
    "\n",
    "for i, word in enumerate(top_negative_words):\n",
    "    print(f\"#{i}\\t{word.rjust(10, ' ')}\\t(ratio={probability_ratio[bow_vocabulary.index(word)]})\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now lets try something less prehistoric: __Logistic Regression__. Turns out, if you're using silicon instead of an abacus, you can find model weights by optimizing the log-probability of the answer. Though, of course, you don't even need to write it by hand anymore. Let's sklearn it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "bow_model = <YOUR CODE HERE - train a logistic regression>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import roc_auc_score, roc_curve\n",
    "\n",
    "for name, X, y, model in [\n",
    "    ('train', X_train_bow, y_train, bow_model),\n",
    "    ('test ', X_test_bow, y_test, bow_model)\n",
    "]:\n",
    "    proba = model.predict_proba(X)[:, 1]\n",
    "    auc = roc_auc_score(y, proba)\n",
    "    plt.plot(*roc_curve(y, proba)[:2], label='%s AUC=%.4f' % (name, auc))\n",
    "\n",
    "plt.plot([0, 1], [0, 1], '--', color='black',)\n",
    "plt.legend(fontsize='large')\n",
    "plt.grid()\n",
    "\n",
    "test_accuracy = np.mean(bow_model.predict(X_test_bow) == y_test)\n",
    "print(f\"Model accuracy: {test_accuracy:.3f}\")\n",
    "assert test_accuracy > 0.77, \"Hint: tune the parameter C to improve performance\"\n",
    "print(\"Well done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task: implement TF-IDF features\n",
    "\n",
    "Not all words are equally useful. One can prioritize rare words and downscale words like \"and\"/\"or\" by using __tf-idf features__. This abbreviation stands for __text frequency/inverse document frequence__ and means exactly that:\n",
    "\n",
    "$$ \\text{feature}_i = \\frac{\\text{Count}(word_i \\in x)}{\\text{Total number of words in } x} \\times \\log\\left(\\frac{N}{\\text{Count}(word_i \\in D) + \\alpha}\\right) $$\n",
    "\n",
    "\n",
    ", where x is a single text, D is your dataset (a collection of texts), N is a total number of documents and $\\alpha$ is a smoothing hyperparameter (typically 1). \n",
    "And $Count(word_i \\in D)$ is the number of documents where $word_i$ appears.\n",
    "\n",
    "It may also be a good idea to normalize each data sample after computing tf-idf features.\n",
    "\n",
    "__Your task:__ implement tf-idf features, train a model and evaluate ROC curve. Compare it with basic BagOfWords model from above.\n",
    "\n",
    "Please don't use sklearn/nltk builtin tf-idf vectorizers in your solution :) You can still use 'em for debugging though."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "\n",
    "```\n",
    "\n",
    "```\n",
    "\n",
    "```\n",
    "\n",
    "```\n",
    "\n",
    "```\n",
    "\n",
    "```\n",
    "\n",
    "```\n",
    "\n",
    "```\n",
    "\n",
    "```\n",
    "\n",
    "```\n",
    "\n",
    "```\n",
    "\n",
    "```\n",
    "\n",
    "```\n",
    "\n",
    "```\n",
    "\n",
    "```\n",
    "\n",
    "\n",
    "### Solving it better: word vectors\n",
    "\n",
    "Let's try another approach: instead of counting per-word frequencies, we shall map all words to pre-trained word vectors and average over them to get text features.\n",
    "\n",
    "This should give us two key advantages: (1) we now have 10^2 features instead of 10^4 and (2) our model can generalize to word that are not in training dataset.\n",
    "\n",
    "We begin with a standard approach with pre-trained word vectors. However, you may also try\n",
    "* training embeddings from scratch on relevant (unlabeled) data\n",
    "* multiplying word vectors by inverse word frequency in dataset (like tf-idf).\n",
    "* concatenating several embeddings\n",
    "    * call `gensim.downloader.info()['models'].keys()` to get a list of available models\n",
    "* clusterizing words by their word-vectors and try bag of cluster_ids\n",
    "\n",
    "__Note:__ loading pre-trained model may take a while. It's a perfect opportunity to refill your cup of tea/coffee and grab some extra cookies. Or binge-watch some tv series if you're slow on internet connection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "import gensim.downloader \n",
    "embeddings = gensim.downloader.load(\"fasttext-wiki-news-subwords-300\")\n",
    "\n",
    "# If you're low on RAM or download speed, use \"glove-wiki-gigaword-100\" instead. Ignore all further asserts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def vectorize_sum(comment):\n",
    "    \"\"\"\n",
    "    implement a function that converts preprocessed comment to a sum of token vectors\n",
    "    \"\"\"\n",
    "    embedding_dim = embeddings.vectors.shape[1]\n",
    "    features = np.zeros([embedding_dim], dtype='float32')\n",
    "    \n",
    "    <YOUR CODE>\n",
    "    \n",
    "    return features\n",
    "\n",
    "assert np.allclose(\n",
    "    vectorize_sum(\"who cares anymore . they attack with impunity .\")[::70],\n",
    "    np.array([ 0.0108616 ,  0.0261663 ,  0.13855131, -0.18510573, -0.46380025])\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train_wv = np.stack([vectorize_sum(text) for text in texts_train])\n",
    "X_test_wv = np.stack([vectorize_sum(text) for text in texts_test])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XdYleUbwPHvy0j2cKEgghtxlyM3Zm4tzZngxD0yrZ9p\nWmCZoWWuXKSWaK4sc+RITdDMyokpLkQQREGR6UDG8/vjxAmUcYADBzjP57q44Jx33e8Rb97znOe9\nb0UIgSRJklS2GOg6AEmSJEn7ZHKXJEkqg2RylyRJKoNkcpckSSqDZHKXJEkqg2RylyRJKoNkcpck\nSSqDZHKXJEkqg2RylyRJKoOMdHXgihUrCmdn5wJt++jRI8zNzbUbUAknz1k/yHPWD4U557Nnzz4Q\nQlTKaz2dJXdnZ2fOnDlToG39/f1xc3PTbkAlnDxn/SDPWT8U5pwVRQnTZD05LCNJklQGyeQuSZJU\nBsnkLkmSVAbJ5C5JklQGyeQuSZJUBuWZ3BVF2aAoSrSiKJdyWK4oirJcUZRgRVEuKorysvbDlCRJ\nkvJDkyv374DuuSzvAdT592scsLrwYUmSJEmFoWjSZk9RFGdgnxCiYTbL1gL+Qoit/z6+BrgJIe7m\nts/mzZuLgsxz3/LXbTb6X8bGxkb9XKzhceIN/873vkqT1NRUjIx0dluCTshz1g9l7Zw7XoildVB8\ntsuepqXyOPkpVLfDc+vxAu1fUZSzQojmea2njVfUAQjP9Dji3+deSO6KooxDdXWPnZ0d/v7++T7Y\nqovHiLM6zwMDRf3cM+MQAF56VjPf+ysthBCkpqbqOoxiJc+5ZOl0MY42VxK1vl8BKHmuVVS030O6\nfsRTAEKrZT2rCw8fsfzqXcwNDZhbzbpA+S8/tJHcs/t3yfYVE0L4Ar6gunIvyB1aKbdWYMBdGlfN\n/CaiPD1r9mRg3YH53l9pIe/iKxtit+8gYd++HJfHxcVhY2NdjBFp7vHp6wCYtWih1f2qztkm7xUL\nKj0Fkp/7o/QkFhJyHVwouEpg5fSE+l1bARD3KIX/bb/MuvO3qW1nzroxLTBt7UnLIv7d1kZyjwAc\nMz2uBkRqYb85Mk6159vu3xblISRJY3kl7Mwenz4NaD9BFgezFi2w6t0b28GDtLrfiF9/pglnIC1F\nq/tV+2tNzsvazQATK+0eTzGExoPAsgppaWm0adSIa9fCmTlzJt7e3piamhb5VTtoJ7nvAaYoirIN\naAXE5zXeLkmlhSaJOz8JO68Eecvfnyal8d3K03hIeZL7Oj9PgvgIULK+2W97/+p/D0yK4F2LsTnY\nuULPL7I+b1YRbByz36aQYmJiKC8EhoaGfPbZZzg6OtK8eZ7D5FqVZ3JXFGUr4AZUVBQlAvACjAGE\nEGuA/UBPIBh4DIwqqmClsiWvxGkbF0fY+g3FGNGLNEncRXVFW2rEhcPyppCu4WcFrm9meRhNeSrX\naAjdF4JB6b71RgjB999/z7Rp0/Dx8WHs2LH069dPJ7HkmdyFEG/nsVwAk7UWkVQmaPuKV1f0PnHn\nJeoy/DZfldibe0KVFybU/UcxgHq9wCJrtdogf38ql8Z3K88JDw9nwoQJ7N+/n1dffZW2bdvqNJ6y\nM/9IKlZ5JW9tXPGW2iEKfXJ5F1zbD5VcoPVkqFBL1xHpxNatWxk/fjxpaWksXbqUKVOmYGhoqNOY\nZHKXslXY5C2vePVAxFm4c051RT75L11Ho1O2tra0atUKX19fatSooetwAJncpRwk7NvH06tXMXFx\nyXa5TN4Se6dB1D9gXV3XkRS71NRUlixZwrNnz5gzZw7du3enW7duKIruZuw/TyZ36QWx23fw+PRp\nzFq0wGmTn67DkZ738BY8elB8x0t9Apv7gxBg+NJ/z6c8Vo2hD9SvacmBgYF4enpy9uxZBg0ahBAC\nRVFKVGIHmdz1RkHmYlv17l2UIZVtCXchcAukp+drM6fQWxBwOucVkhPgj+WFDK6ArKpBg75Zn2vw\nFhiV0008xSw5OZn58+fj4+ND+fLl+eGHH+jfv3+JS+oZZHIvQ3JL4Nqciy1p4ML38Nun+d6sBkCo\nBiu+Oglqdc73/gvM0Bic2qi+66kbN26wcOFChg4dyldffUWFChV0HVKuZHIvI2K37+CelxeQfQKX\nCbuYiX+v2OdGq+5Y1FBAQAAdO3bMfSVFAQPdzsTQF0lJSezevRt3d3caNmzI1atXqVmzdNSwksm9\nFMt8pZ5xZV5l3jyZwEsSA6N8JWJhYAiG8r9lSXD48GHGjRtHWFgYL7/8MvXr1y81iR1kci+1nr9S\nl1fmxeBpgmZ3YR7/Ev75oejjkYpEbGws77//Phs2bKBu3boEBARQv359XYeVbzK5lzA5jZs/fyu+\nvFIvIncDISb4xefD/oDT6/K3rzbvyOGTUiYtLY22bdty/fp1Zs+ezccff4yJiYmuwyoQmdxLgOyG\nV/L64FNeqWvR3UA4vR4QcC6PqZ/t3wMLu7z3WbMTVKqrlfCkovfgwQPKly+PoaEhCxYsoHr16rz8\ncunuGCqTezHIz92eOSVteSt+Edk9Bc5vUv1sWRXMKkDz0dAom94AJtZgWaV445OKlBCCTZs28e67\n7+Lj48O4cePo27dv3huWAjK5F5H8XI3Lq/B8iA2Dm0e1t78re8C2Brw6EVqN195+pRIvLCyM8ePH\nc+jQIdq0aUOHDh10HZJWyeReRDLfvi+TdyGEnoTT36jujgQI+ln7x2g5TiZ2PbN582YmTpyIEIIV\nK1YwadIkDEp5ueHnyeRehExcXOTt+5p6eEvV8CGTFn9PBv8I1YOK9f79XhccXoHXvbV0YAUsKmtp\nX1JpUalSJdq2bcvatWtxcnLSdThFQib3IpC5NouUh4S7cOLLbGeimGf88NY3qrZlklRAKSkpLF68\nmJSUFD766CO6detG165dS2zpAG2Qyb0IZIy1y9osGjjn919i7/gBVG2qXvTPpUs06j2+aFqvSXrj\n/PnzeHp6cv78eYYMGVJiC31pm0zuRcSsRQs5xv68pPsQHZT1uYc3Vd/nRr9QgCrmnplM7FKBPX36\nlE8++YRFixZRsWJFfvzxR9566y1dh1VsZHKXis9PYyDE/8Xny1npTWVBqfgEBwfz5ZdfMnz4cBYv\nXoytra2uQypWMrlLRef0Oriw5b/H0VfBvhl0/Szrelb2xRuXVGYlJSWxa9cuhg0bRsOGDbl27VqJ\n6YxU3GRyL6TsblDKrYORXrmyFx4Eg+O/Hyw7tYbGQ8BZt42DpbLp0KFDjBs3jvDwcJo3b079+vX1\nNrGDTO6Fll07OhMXF/38MDX0JFz+6b/H969DZRfw+FF3MUllXkxMDDNmzMDPzw8XFxdOnDhRKgt9\naZtM7lqgN/PZhYCESEC8+PyPYyD8T9Vjs0xNDBxbFlt4kv7JKPQVHBzMnDlzmDt3bqkt9KVtMrnn\nQ3ZDMHo1nz1gIfh/nvs6fVdD06HFE4+kt+7fv0+FChUwNDRk4cKFODk50bRp07w31CMyuedDdkMw\nGaUF9EJSFLxkCd0XvLhMMYR6PcCsfPHHJekNIQTfffcdM2bMwMfHh/Hjx/Pmm2/qOqwSSSZ3DWRc\nsWckdr0YggFIT4PQE/DDSEBRdbsvZwUvD9d1ZJIeCg0NZdy4cRw+fJj27dvTqVMnXYdUosnknofn\nOx7pzVV62Cn40RMS7qgeV3YFx1aqui6SVMw2bdrExIkTURSFVatWMX78+DJX6EvbZHLPQ8YYu950\nPEp5CnvfgYvb/3vO/Ueo1Ul2FZJ0xs7Ojg4dOrBmzRqqV6+u63BKBZncNaA3pQQu7/p3COZfvRbD\nyyPA0FhnIUn6KSUlhUWLFpGWlsbHH39M165d6dq1q67DKlVkcpdUQgLg738LeLnNhlcngYmVbmOS\n9NK5c+cYPXo0gYGBDB06VF3oS8ofjQatFEXprijKNUVRghVFmZXN8uqKohxTFOW8oigXFUXpqf1Q\npSKRnARbBoPfGxD2u6peescPZGKXit2TJ0+YNWsWLVu2JCoqil27dvH999/LxF5AeV65K4piCKwE\nugARwGlFUfYIITKX95sL7BBCrFYUxRXYDzgXQbySNgXtgR3D/nvcfz00GqC7eCS9FhISwldffcXI\nkSP54osv9K7Ql7ZpMizTEggWQoQAKIqyDXgTyJzcBZBxqWcNRGozSEkLQn+Hq79kfS7ygup7Zy9o\nMUZerUvFLiEhgYMHD+Lm5kaDBg24ceNGme2MVNw0Se4OQHimxxFAq+fW8QZ+VRRlKqoGOq9rJTpJ\ne35fAjd/g5cssj5v1wjavAOG8uMXqXjt37+fCRMmcOfOHUaMGEH9+vVlYtciTf5HZzfg9VxxEd4G\nvhNCLFYUpTWwSVGUhkKI9Cw7UpRxwDhQTW3y9/fPd8CpqakIIQq0bX6ZnjiB1enTPKtTh1vFcLzc\nJCUlFeqcG8fEYGRRm3OvfPHiwhO/FzywIlTYcy6N9OGc4+PjWblyJYcPH8bJyYmFCxcSFRVFVFSU\nrkMrNsXx76xJco8AHDM9rsaLwy6eQHcAIcQpRVFMgIpAdOaVhBC+gC9A8+bNhZubW/4DvrWC1NRU\nCrJtfoWt38BjoLqHB02K4Xi58ff3z/85J0ZBgA+kJkPqPbByKJbXTVsKdM6lXFk/57S0NFxdXQkJ\nCeHjjz/mww8/5NSpU2X6nLNTHP/OmiT300AdRVFqAHeAIcDzlaFuA52B7xRFqQ+YAPe1GaiulNo5\n7n+uhoMZE5sUsK4GNTroNCRJf0VFRVGpUiUMDQ358ssvcXJyonHjxroOq0zLcyqkECIVmAIcAq6g\nmhVzWVGUTxRFeePf1d4DxiqKEghsBUYKIZ4fupGKU9gfqv6jbafBxzEw/RK87qXrqCQ9I4Rg/fr1\n1KtXD19fXwD69OkjE3sx0OhTNCHEflTTGzM/93Gmn4MA2V5HV45/8d/MlwwRZ8DSHrp8opuYJL0X\nEhLC2LFj+e233+jYsSOvvy7nWRQnOUWiLDi5HAyMwLLqf8+ZVVCV4JUkHdi4cSOTJk3C0NCQNWvW\nMHbsWFnoq5jJ5F5WNBkC3fNopCFJxcTe3p7XXnuN1atXU61aNV2Ho5dkci/Nku7DninwLEnXkUh6\n7tmzZ/j4+JCeno63tzddunShS5cuug5Lr8n3SaXVX2vhy9pw/SAYm0Ed+R9J0o3Tp0/zyiuv4OXl\nRUhICHIuRckgk3s2YrfvIGzYcJ5evarrUHL24AYYmUK3BTDzFtR6TdcRSXrm8ePHvP/++7z66qvE\nxsayZ88e/Pz8ZKGvEkIm92xkbqlXIjsvHfwQLv8ExqbQejIYvaTriCQ9dOvWLVasWMHYsWO5fPky\nffr00XVIUiZyzP05sdt38Pj0acxatCiZvVJPr4M/V6p+fm2ubmOR9E58fDw//fQTo0aNokGDBgQH\nB+Po6Jj3hlKxk8k9k8z9UkvEFXtaKhybD09iqRt5F0IWwu0/oGI98NgJNrLdmFR8fvnlF8aPH8/d\nu3dp3bo1Li4uMrGXYHJYJpMS1y/14U1VNcdLu6gQ8zc8uA62zvD2VpnYpWJz//593N3d6d27N7a2\ntpw6dQoXFxddhyXlQV65P6dE1ZLJmHXQZymnHpTXu+JKku6lpaXRrl07bt26xbx585g1axYvvSQ/\n4ykNZHIvqaKC4Ii3rqOQ9NS9e/eoXLkyhoaGLF68GGdnZxo2bKjrsKR8kMMyJdHTBDizHm4cgiqN\nVF+SVAzS09NZu3YtdevWZe3atQD07t1bJvZSSF65l0Tb3eHWcTAsB+MCwMAQVbVlSSo6wcHBjB07\nFn9/f1577TW6deum65CkQpDJXZfS02HfNIh/LnHfOQ9Vm0KfZf8mdkkqWt9++y2TJk3ipZde4ptv\nvsHT01PejFTKyeSuK+lpEHIMzvmBtSNY2P23rFI9eGUk2DfVWXiSfqlevTrdunVj5cqVODg46Doc\nSQtkcteFiLPw8wTV1EYAt1nQzEO3MUl6JTk5mc8//5z09HQ++eQTOnfuTOfOnXUdlqRFMrkXByEg\n5bHq++7JEPTzf8s8fpLt76Ri9ddff+Hp6cnly5cZMWIEQgg5BFMGyeSO6s7UzPVktOreJdg1AaL+\nyfp8r6+ghad2jyVJuXj06BEfffQRS5cuxcHBgX379tGrVy9dhyUVEZncKaJCYU8T4MSXcHLZf891\n+UTVManRILCopJ3jSJKGwsLCWLVqFRMmTMDHxwcrKytdhyQVIb1P7kVWKOz2n6rEbvgStJ6ialRt\naqO9/UuSBuLi4ti5cydjxozB1dWV4OBg2RlJT+h1ci+yQmGJ9yDEX/Xz6IPg8Ir29i1JGtq9ezcT\nJ04kOjqadu3a4eLiIhO7HtHrO1SLrFBYwKJ/y/IqYFpee/uVJA1ER0czZMgQ+vbtS6VKlfjzzz9l\noS89pNdX7lBEhcLSklXz1if+AeYVtbtvScpFWloabdu25fbt28yfP5+ZM2dibGys67AkHdD75F5k\nDIxkYpeKTWRkJFWqVMHQ0JBly5bh7OyMq6urrsOSdEivh2WKRMAXcNNf11FIeiI9PZ3Vq1fj4uLC\nmjVrAOjZs6dM7JJM7lojBERdVs2QSX0CDfrpOiKpjLt+/TqdOnVi0qRJtGrVih49eug6JKkEkcld\nW+5dhNVt4FkivDwCun2m64ikMmz9+vU0adKEixcvsmHDBn799Vdq1Kih67CkEkQvx9y1fkdq8FE4\nrJpSSZdPofnowu9TknLh7OxMjx49WLlyJVWrVtV1OFIJpHfJPfPcdrMWLQo/v/1hCJxeB9FBUK8n\nNHWHchZaiFSS/pOcnMynn34KwPz582WhLylPepfctT63fctgVXVHa0dV42pJ0rI//vgDT09Prl69\nyujRo2WhL0kjejnmrtW57c8eQ93uMOaodvYnSf9KSkpi2rRptGvXjsePH3Pw4EHWr18vE7ukEY2S\nu6Io3RVFuaYoSrCiKLNyWGeQoihBiqJcVhRli3bDLOHMKoKlXd7rSVI+3L59m7Vr1zJ58mQuXbok\n295J+ZJnclcUxRBYCfQAXIG3FUVxfW6dOsBsoK0QogHwbhHEWmgZRcIkqaRKTEzE19cXAFdXV0JC\nQlixYgWWlpY6jkwqbTQZc28JBAshQgAURdkGvAkEZVpnLLBSCBELIISI1nag2pAx3q7VImGSpCW7\ndu1izJgxxMfH07FjR+rVq4e9vb2uw5JKKU2SuwMQnulxBNDquXXqAiiKchIwBLyFEAef35GiKOOA\ncQB2dnb4+/vnO+DU1FSEEAXa1jYuDurUIdCuMhRg++y8mvyU2Ht3uaal/eUkKSmpQOdcmunLOT98\n+JDly5cTEBBAzZo1+fzzz7l79y53797VdWjFQl/+nTMrjnPWJLln9+mNyGY/dQA3oBpwQlGUhkKI\nuCwbCeEL+AI0b95cuLm55TdejG6tIDU1lYJsG7Z+AwBNCrBtjs6ZULVKVapqc5/Z8Pf3L9A5l2b6\ncM5paWm4uLgQHh7OggULaNGiBa+//rquwypW+vDv/LziOGdNPlCNABwzPa4GRGazzm4hRIoQ4hZw\nDVWyL7uS7sOSRpAQkf2fP0nKRUREBOnp6RgaGrJ8+XIuXLjA7NmzMTLSu9nJUhHRJLmfBuooilJD\nUZSXgCHAnufW+RnoBKAoSkVUwzQh2gy0xEmIgPjbqhuXWozVdTRSKZGens6KFStwcXFh9erVAPTo\n0UPWW5e0Ls/kLoRIBaYAh4ArwA4hxGVFUT5RFOWNf1c7BMQoihIEHAP+J4SIKaqgS5SXh4N9U11H\nIZUCV69epUOHDrzzzju0a9eO3vKDfakIafQeUAixH9j/3HMfZ/pZADP+/ZIk6Tnr1q1jypQpmJmZ\nsXHjRoYNGyZvRpKKlBzgk6RiUKtWLfr06cPXX3+NnZ284U0qejK5S1IRePr0KZ988gkACxYsoFOn\nTnTq1EnHUUn6RC9ry0hSUTp58iRNmzbl888/5/79+6hGLSWpeMnkLklakpiYyNSpU2nfvj3Jyckc\nOnSIb775Ro6tSzohk7skaUlERATr1q1j6tSp/PPPP3Tt2lXXIUl6TI65S1IhxMTEsGPHDiZOnEj9\n+vUJCQmRnZGkEkFeuUtSAQgh2LlzJ66urrzzzjtcu3YNQCZ2qcSQyb0ggvbApn7/PpDjqfrm7t27\n9O/fn4EDB+Lo6MiZM2eoV6+ersOSpCz0YlhG6w2x7wbCk1hoMxWcWhd+f1KpkZaWRvv27blz5w6L\nFi1i+vTpsh6MVCLpxW9l5sRe6Fruf66BSztBMYSu87UToFTihYeH4+DggKGhIStXrqRGjRrUrVtX\n12FJUo70ZljGxMUFp01+he+demYDPImDpm9rJzCpREtLS2P58uVZCn1169ZNJnapxNOLK3etq+kG\nb67UdRRSEbty5Qqenp6cOnWKHj160KdPH12HJEka05srd61ISwGRrusopGLg6+tL06ZNuX79Ops2\nbeKXX36hevXqug5LkjQmk7smUp5A4Hb4tCLE3AAD+YanrKtTpw79+vUjKCgIDw8PeZepVOrILKWJ\nsxvh4Aeqn53bQ8eZuo1H0ronT57g7e2Noij4+PjIQl9SqSev3DWR8lj1ffxxGLEXKsk5zWXJ8ePH\nadKkCYsWLSI+Pl4W+pLKBJnc83L/umpeO0DFeiDfnpcZCQkJTJo0iY4dO5KWlsbRo0dZvXq1HIKR\nyoQyn9xjt+/g8enT+d8wPR0Oe8HKFhD0M5hYy7H2MiYyMpLvvvuOGTNmcPHiRV577TVdhyRJWlPm\ns1XCvn0A+b95KekenFwKJjbQ8C3o7AWGZf7lKvMePHjAjh07mDRpEi4uLty6dUt2RpLKpDJ/5Q5g\n1qJFwW9e6jIPei8BUxvtBiUVKyEE27dvx9XVlXfffZfr168DyMQulVl6kdwl/RYZGUnfvn0ZMmQI\nTk5OnD17Vt5hKpV5cpxBKtPS0tLo0KEDd+7c4csvv2TatGmy0JekF8r0b3nGh6lmLVroOhSpmIWF\nhVGtWjUMDQ1ZtWoVNWvWpHbt2roOS5KKTZkelinwh6lSqZWWlsZXX31F/fr11YW+unbtKhO7pHfK\n9JU7FPLDVKlUuXTpEp6envz999/07t2bvn376jokSdKZMp/c8+3WCbh+EJ4l6ToSKR/WrFnDO++8\ng7W1NVu2bGHIkCHyZiRJr8nk/rwTX0JIABibqea4V5SzKkoyIQSKolC/fn0GDhzI0qVLqVSpkq7D\nkiSdk8kdIDUZbhyGtGRIjILqr8Log7qOSsrF48eP+fjjjzE0NGThwoV07NiRjh076josSSoxyuwH\nqvkqO3BlL2x3h52j4f4VMC1ftMFJheLv70/jxo1ZvHgxSUlJstCXJGWjzF6552umTGqy6vuwn8HK\nHmxkU4aSKD4+npkzZ+Lr60utWrX47bffZFleScqBRlfuiqJ0VxTlmqIowYqizMplvQGKoghFUZpr\nL8T8id2+g7Bhw3l69Wr+Z8qUr6kq52tsWnQBSgV29+5dNm/ezPvvv8/FixdlYpekXOR55a4oiiGw\nEugCRACnFUXZI4QIem49S+Ad4K+iCFRTCfv28fTqVUxcXOT89jLg/v37bNu2jalTp+Li4kJoaKj8\nwFSSNKDJsExLIFgIEQKgKMo24E0g6Ln1PgUWAe9rNcICMHFxwWmTn2YrP3sMcbeLNiAp34QQHDly\nhAEDBpCQkEC3bt2oW7euTOySpCFNhmUcgPBMjyP+fU5NUZRmgKMQYp8WYysev8yAAB/Vz3I4pkQI\nDw+nT58+fPbZZ9SuXZvz58/LQl+SlE+aXLlndyeIenqCoigGwBJgZJ47UpRxwDhQlVr19/fXKMjM\nUlNTEULg7++P6YkTmPyddUaMUUQEqdWqcUvDfTeMuIm5iR1X6r9HwpkgXnxDUjIkJSUV6PUqbdLS\n0hg+fDgPHz5kzJgxDBkyhPv37+vFuYP+/DtnJs+5aGiS3CMAx0yPqwGRmR5bAg0B/3/vCKwC7FEU\n5Q0hxJnMOxJC+AK+AM2bNxdubm75D/jWClJTU3FzcyNs/Qae3ruHiYvLfyvY2GDVuzdNNN333bVg\n/ISX3xyf71iKk7+/PwV5vUqL0NBQHB0dMTQ0ZOPGjdSsWZPbt2+X6XPOTln/d86OPOeioUlyPw3U\nURSlBnAHGAIMzVgohIgHKmY8VhTFH3j/+cReVPI1vi6VOKmpqSxdupSPPvqIRYsWMXXqVF5//XUA\nbt+Wn4VIUkHlmdyFEKmKokwBDgGGwAYhxGVFUT4Bzggh9hR1kFLZdPHiRTw9PTlz5gxvvvkm/fv3\nL9T+EhISiI6OJiUlRUsRFj9ra2uuXLmi6zCKlTznrIyNjalcuTJWVlaFOoZGNzEJIfYD+5977uMc\n1nUrVETFKfICJN7TdRR6adWqVUybNg1bW1u2b9/OwIEDC1XoKyEhgaioKBwcHDA1NS21RcMSExOx\ntLTUdRjFSp7zf4QQPHnyhDt37gAUKsGX2fIDeUpPh/VdIfKcqkCYVCwySgU0bNiQIUOGEBQUxKBB\ngwqdjKOjo3FwcMDMzKzUJnZJUhQFMzMzHBwciI6OLtS+ymz5gbwJVaGw5p6qJthSkXr06BFz587F\nyMiIL774gg4dOtChQwet7T8lJQVTUzmVVSobTE1NCz28qJ9X7qnJcOkn1c+WVaCcfr0lLG5Hjx6l\nUaNGLF26lOTk5CIr9CWv2KWyQhu/y/qZ3G/+Bj+NUf1sLu94LCpxcXGMGTOG119/HSMjI44fP87y\n5cv1Mgk7Oztz5MgRXYeRbxYWFoSEhOg6DKkA9DO5Z1SBHL4bXhmp01DKsqioKLZt28YHH3xAYGAg\n7du313VIesPf359q1aoVej9JSUnUrFmzwNs/evQICwsLevbs+cIyRVEIDg7O8py3tzceHh7qxwkJ\nCbz77rtUr14dCwsLateuzbvvvsuDBw/yFcfXX39N8+bNKVeuHCNHjsxz/SVLllClShWsra0ZPXo0\nycnJ6mWhoaF06tQJMzMzXFxcXvijrcm2dnZ22W6rTfqZ3DOYVwY9vIosSlFRUSxbtgyAevXqERoa\nio+Pjxy9BQTJAAAgAElEQVQPL4FSU1OL/Bg7d+6kXLly/Prrr9y9ezdf2z579ozOnTtz+fJlDh48\nSEJCAn/88QcVKlTg77//zte+7O3tmTt3LqNHj85z3UOHDuHj48PRo0cJDQ0lJCQELy8v9fK3336b\nZs2aERMTw2effcaAAQO4f/9+vrYNDQ19YVtt0+/kLmmNEILNmzfj6urKzJkzuXHjBgAVK1bMY0v9\ncfr0aVxdXbG1tWXUqFE8ffpUveybb76hdu3aVK9enTfeeIPISNVN4F5eXkydOhVQfWhsbm7OzJkz\nAXjy5AkmJibExsZmOc6jR4/o0aMHkZGRWFhYYGFhQWRkJN7e3gwYMAAPDw+srKz47rvv+Pvvv2nd\nujU2NjZUrVqVKVOm8OzZM/W+Ml9djxw5ksmTJ9OrVy8sLS1p1aoVN2/ezPWcN27cyIQJE2jcuDHf\nf/99vl4vPz8/bt++za5du3B1dcXAwIDKlSvz0UcfZftOIDdvvfUWffv2pUKFCnmuu3HjRjw9PWnQ\noAG2trZ89NFHfPfddwBcv36dc+fOMW/ePExNTenfvz+NGjXixx9/LPS22qZ/yf33JfCjp+pnRf9O\nvyjcvn2bXr16MWzYMOrVq8eFCxeoU6eOrsMqcb7//nsOHTrEzZs3uX79OvPnzwfgt99+Y/bs2ezY\nsYMbN27g5OTEkCFDAOjYsaO6Bsnp06epUqUKAQEBAJw6dYp69epha2ub5Tjm5uYcOHAAe3t7kpKS\nSEpKwt7eHoDdu3czYMAA4uLicHd3x9DQkCVLlvDgwQNOnTrF0aNHWbVqVY7nsHXrVry8vIiNjaV2\n7drMmTMnx3Vv376Nv78/7u7uuLu74+eXvzvJjxw5Qvfu3bGwsMhxnd69e2NjY5PtV+8Clvy+fPky\nTZo0UT9u0qQJUVFRxMTEcPnyZWrWrJlljnqTJk24fPlyobfVNv2bChl1GV4yh3YzZPNrLcio8xMd\nHc3y5cuZNGkShoaGug6LeXsvExSZUKTHcLW3wqtPA43XnzJlCo6OqjJNc+bMYerUqcyfP5/vv/+e\n0aNH8/LLL5OYmMjnn3+Ora0toaGhtG7dmhs3bhATE8Px48fx9PRk1apVJCUlERAQkO++sa1bt6Zv\n376AarrdK6+8ol7m7OzM+PHjCQgI4N133812+7feeouWLVsC4O7uzowZM3I8lp+fH40bN8bV1RUb\nGxtmzpzJ+fPnadasmUaxxsTEZIkvO/v2ab8QbVJSEtbW1urHGT8nJia+sCxjecZNR4XZVtv089LV\nrAK0excM9PP0tSEkJIS0tDSMjIz45ptvuHTpElOnTi0Rib2kykjsAE5OTuqhl8jISJycnNTLLCws\nqFChAnfu3MHU1JTmzZsTEBDA8ePH6dixI23atOHkyZMFSu6ZYwDVUEHv3r2pUqUKVlZWfPjhh7l+\nWFmlShX1z2ZmZiQlJeW4rp+fH+7u7oBqzLtjx45s3LhRvdzQ0PCFudwpKSkYGxsDUKFChXyP02uD\nhYUFCQn/XRhk/GxpafnCsozlGVfjhdlW2/Tjyj09HcJ+VzXmSIjMe30pR6mpqSxevBgvLy8WLVrE\nO++8Q+fOnXUd1gvyc0VdXMLD/2uLcPv2bfVQib29PWFhYepljx49IiYmBgcHVduEjh078ttvv3H+\n/HlatGhBx44dOXToEH///XeON4LlNN30+ecnTpxIs2bN2Lp1K5aWlixdupSdO3cW6jwB/vjjD27c\nuMHnn3/O4sWLAdXV6+XLl/nyyy8xMjKievXqhIaGZpnVc+vWLXXt/tdff525c+fy6NEjzM3Nsz1O\njx49OHHiRLbL2rdvz4EDB/Ide4MGDQgMDGTQIFWLzsDAQOzs7KhQoQINGjQgJCQkS/mAwMBAhg4d\nmq9tM2TeVtv049L19inY2Ae2Doawk2Binfc20gsuXLhAq1atmDVrFj179mTgwIG6DqlUWblyJRER\nETx8+JAFCxYwePBgAIYOHcq3337LhQsXSE5O5sMPP6RVq1Y4OzsDquTu5+eHq6srL730Em5ubqxb\nt44aNWrk2JnKzs6OmJgY4uPjc40pMTERKysrLCwsuHr1KqtXr9bKuW7cuJEuXboQFBTEhQsXuHDh\nApcuXeLx48fqhDt48GDmz5/PnTt3SE9P58iRI+zdu5cBAwYAMGzYMBwdHenfvz9Xr14lPT2dmJgY\nFixYwP79qlJXBw4cUH+u8PxX5sSemprK06dPSUtLIy0tjadPn+Y4W2j48OGsX7+eoKAgYmNjmT9/\nvnr6ZN26dWnatCnz5s3j6dOn7Nq1i4sXL6qL3hVmW60TQujk65VXXhEF0XLDW+Jl3zeEEEKEegwT\noR7Dcl55zzQhVr4qxFcNhfCyEuLcJiEizgrxKKZAx9alY8eO6fT4K1asEEZGRsLOzk7s3LmzWI6Z\nn3MOCgoqukC0wMnJSSxYsEDUr19fWFtbi+HDh4tHjx6pl69evVrUrFlT2NjYiF69eonw8HD1ssTE\nRGFkZCS8vb2FEEKkp6eLSpUqiQkTJuR6zFGjRony5csLa2trcefOHeHl5SXc3d2zrBMQECDq1asn\nzM3NRbt27cRHH30k2rZtq14OiBs3bgghhBgxYoSYM2eOetmxY8eEg4PDC8d98uSJsLGxEXv27Hlh\n2cSJE0X//v2FEEI8fvxYvP/++6J69erCyspKNGvWTOzevTvL+nFxcWLatGmiWrVqwtzcXNSsWVNM\nnz5dPHjwINdzf56Xl5dA1WRI/eXl5SWEECIsLEyYm5uLsLAw9fqLFy8WlStXFpaWlmLkyJHi6dOn\n6mW3bt0SHTt2FCYmJqJu3bri8OHDWY5VmG0zy+l3GlU13jxzrCKK6FbwvDRv3lycOZP/ku+tvu1P\namoqZ8fuJmzYcICc67kvqgkvWUDVxlDOGnouUn2YWgrpqqGBEAJFUTh+/DgbNmzgq6++onz58sVy\n7Pyc85UrV6hfv37RBlQMZIVE/aDJOef0O60oylkhRPO8jlH2x9zrdIFei3UdRamTlJTEnDlzMDY2\n5ssvv9R6oS9JkoqWfoy5S/ny66+/0rBhQ1asWEFKSkqRFfqSJKnoyOQuqcXGxjJq1Ci6deuGiYkJ\nx48fZ9myZXpZ6EuSSjuZ3CW16Ohodu7cyezZs7lw4QLt2rXTdUiSJBVQ2R1zv38N0oq+MFJpd+/e\nPbZu3cr06dPVhb40qb8hSVLJVjav3KMuw8qWkBwPxma6jqZEEkKwceNGXF1dmT17trrQl0zsklQ2\nlM3k/iRO9b3zx9Dhf7qNpQQKDQ2le/fujBw5EldXV1noS5LKoLI1LBN3Gw7NgSt7VI+rtQCTgncP\nL4tSU1Pp1KkTDx48YOXKlUyYMAEDWWNHksqcsvW/esvg/xJ718+gWkvdxlOCBAcHqwt9bdiwgUuX\nLjFp0iSZ2ItJaW2zJ5VeZet/9rNHUKszTA+CNlPA2ETXEelcSkoKCxYsoEGDBqxcuRKATp06ZalC\nKJU92mqzB6hr2eTXrVu3MDAwYNKkSVmeDw0NRVGUF2q7jBw5krlz56of3717F09PT6pWrYqlpSUu\nLi54eXnx6NGjfMWRV1u8zB4+fMjgwYOpWLEiFStWxN3dPUslx06dOlGpUiWsrKxo0qQJu3fvVi/z\n9/fHwMBA3SDFwsJCXQUzOTkZT09PnJycsLS0pF27dgUqapYfZSu5g6rhtbWDrqMoEc6dO0fLli2Z\nM2cOb775prpQlSQVBz8/P2xtbdm2bVuWPqKaePjwIa1bt+bJkyecOnWKxMREDh8+TFxcXJ7dn56X\nW1u8582dO5fY2FhCQkK4efMmUVFReHt7q5cvW7aMu3fvkpCQgK+vLx4eHlnKEmdukJKUlMSIESMA\n1XCoo6MjAQEBxMfHM2fOHAYNGkRoaGi+ziU/yl5ylwBYvnw5LVu25N69e/z000/s2LEDOzs7XYel\n13TdZi89PR0fHx9q1apFhQoVGDRoEA8fPgTg6dOneHh4UKFCBWxsbGjRogVRUVHMmTOHEydOMGXK\nFCwsLJgyZYrG5+vn58f8+fMxNjZm7969+XqtvvrqKywtLdm8ebO6OqajoyPLli2jcePGGu8nv63t\nbt26Rd++fbGyssLa2pp+/fpl6ZTUuHFjjIxUH1UqikJKSkqWUs45MTc3x9vbG2dnZwwMDOjRowc1\natTg7NmzGp9LfsnkXsZklApo1qwZw4cPJygoiH79+uk4Kgl032Zv+fLl/PzzzwQEBBAZGYmtrS2T\nJ08GVCV64+PjCQ8PJyYmhjVr1mBqaspnn31G+/bt+frrr0lKSuLrr7/W6FxPnDhBREQEQ4YMYdCg\nQQVqsffWW2/l+plQ48aNc2yxlzEUlN/WdpMnT2bfvn3ExsYSGxvLjz/+SI8ePbKs07t3b0xMTGjV\nqhVubm40b/5fDa/o6Gjs7OyoUaMG06dPz3EIKTo6muvXr9OgQdH1HShbs2X0WGJiIrNnz6ZcuXIs\nXryY9u3b0759e12HpTsHZsG9f4r2GFUaQQ8fjVfXdZu9tWvX8vXXX6vH4r29valevTqbNm3C2NiY\nmJgYgoODady4cZ7t7fKyceNGevToga2tLUOHDqVDhw5ER0dTuXJljbaPiYmhatWqua5z8eLFPPeT\n39Z2L7/8Ms+ePVPf79G5c+cXPjPYt28fKSkpHDlyhKtXr6r/ALm4uHDhwgVcXFwICwtjxIgRzJgx\ng7Vr12bZPiUlhTFjxjBixAhcXFzyPIeCklfuZcDBgwdp2LAhq1atUtdylkoeXbfZCwsLo1+/fuqr\n2/r162NoaEhUVBTDhg2jW7duDBkyBHt7e2bOnPlCCzxNPXnyhB9++EHdYq9169ZUr16dLVu2AKiH\nNYqjxV5+W9sNHDiQunXrkpiYSEJCArVq1cLDw+OF9YyNjenRoweHDh1izx7VDL0qVarg6uqKgYEB\nNWrUYNGiRS90tUpPT2fYsGEYGxtr/C6ooOSVeykWExPDjBkz8PPzo379+pw8eZLWrVvrOqySIR9X\n1MVF1232HB0d2bBhA23bts12Gy8vL7y8vAgNDaVnz57Uq1cPT0/PfBeO27VrFwkJCUyaNEn9eUFc\nXBx+fn68++67VK1aFWNjY0JDQ7PUK7916xZdunQBVC32du3ahZeXV45DMw0aNMjyumXm4eHBmjVr\n8myL97zAwEBWrVqlbus3YcKEXGsspaam5vgBr6IoWS60hBB4enoSFRXF9u3b1X/Iioq8ci/FYmJi\n2LVrFx999BHnz5+Xib2E03WbvQkTJjBnzhx1Qrx//756Kt+xY8f4559/SEtLw8rKCmNjY3Wzczs7\nO0JCQjQ+z40bNzJ69Gj++ecfdYu9kydPcuHCBf755x8MDQ3p378/c+bMISYmhpSUFLZu3UpQUJB6\nfHvGjBkkJCQwYsQIdbx37txhxowZ6uGYy5cv59hib82aNUD+W9u1aNGCdevW8eTJE548eYKvry9N\nmjQB4OrVqxw4cIAnT56QkpLC5s2b1e+mQDUV8vbt2wghCA8PZ9asWbz55pvqfU+cOJErV66wd+9e\nTE1NNX49C0yTdk1Ad+AaEAzMymb5DCAIuAgcBZzy2meRtNlb0kiIH8cVaL8lXUbLucjISPHFF1+I\n9PR0IYQQDx8+1GFURUu22VPRVpu9tLQ0sXjxYlG3bl1hYWEhatasKWbPni2EEGLLli2ibt26wszM\nTFSuXFlMnTpVpKSkCCGE+OOPP0SdOnWEjY2NmDp1aq7HjIiIEIaGhuLixYsvLOvRo4d47733hBCq\n31tPT09hb28vbGxsRJs2bcTvv/+eZf07d+6IUaNGCTs7O2FhYSHq1asnvL29s7xumsittd3mzZuF\nq6ur+nFISIjo3bu3KF++vLC1tRXdunUT169fF0KofsdatmwpLCwshLW1tWjevLn46aef1NsuXrxY\n2NvbC1NTU1GtWjUxZcoUkZCQIIQQIjQ0VACiXLlywtzcXP21efPmHOMubJs9TRK7IXATqAm8BAQC\nrs+t0wkw+/fnicD2vPYrk3v+/Pbbb2L9+vXC2tpamJiYqH/hyrKylNw1lZEM9Ik85+wVNrlrMizT\nEggWQoQIIZ4B24A3M68ghDgmhHj878M/Ae3cGicBqrHI//3vf3h6etKkSRMCAwNloS9JknKlyQeq\nDkDmWfoRQKtc1vcEsr2vVlGUccA4UI3jZczfzY/U1FSEEPj7+2Mbp6r+eOvf/bR6+pT4qHtcLcB+\nS6q0tDQ8PDyIj49n+vTp9O7dm8jISPVMi7IsKSlJ498Ra2trEhMTizagYpCWllYmziM/5Dln7+nT\npwXKkRk0Se7ZfVSe7Vw7RVE8gOZAtvOzhBC+gC9A8+bNhaad7TMzurWC1NRU3NzcCFu/AYAmGfu5\nYIKpXRWqFGC/Jc2NGzeoWbMmhoaGbN26lejoaAYNGqTrsIqVv78/mv6OXLlyJc9u8qVB5lkd+kKe\nc/ZMTExo1qxZgY+hybBMBOCY6XE14IXLRkVRXgfmAG8IIfJXSKKw0lLgt8/g8cNiPWxRSElJYf78\n+TRs2FA9D9bNzU3jmz8kSZJAsyv300AdRVFqAHeAIUCWSaKKojQD1gLdhRDRWo8yL/evwvFF8JIF\nOBTuzjpdOnPmDJ6enly8eJEhQ4bw9ttv6zokSZJKqTyv3IUQqcAU4BBwBdghhLisKMoniqK88e9q\nXwAWwA+KolxQFGVPkUWcfZCq7/3WQqtxxXpobVm2bBmtWrXiwYMH7N69m61bt8qrdUmSCkyjO1SF\nEPuB/c8993Gmn1/Xclx6QwiBoig0b94cT09PFi1ahI2Nja7DkiSplJPlB3QkISGBDz74ABMTE5Ys\nWULbtm1zvC1ckiQpv2T5AR3Yv38/DRo0wNfXFyMjI1noSyoxevTooe4eJJVuMrkXowcPHuDh4UGv\nXr2wtrbmjz/+4Isvvsh3YSZJyo6iKAQHBxdqHwcOHFB3DyooNzc3bG1tX+i+lF27Pn9//yxlb4UQ\nLF++nIYNG2Jubk61atUYOHAg//yTv/LNQgg++OADKlSoQIUKFZg5c2aOF1FCCD777DOqV6+OlZUV\nQ4YMyVJJ8s6dO7z55puUL1+eatWqqevWZNi7dy8NGzbEwsKCNm3aEBQUpF6WnJzM9OnTsbe3x9bW\nlkmTJhW42mZ+ldrkHrt9B49Pn9Z1GPkSGxvL3r178fLy4ty5c7Rqldu9YJKkXc/3LC0KoaGhnDhx\nAkVR1KVw82PatGksW7aM5cuX8/DhQ65fv07fvn355Zdf8rUfX19ffv75ZwIDA7l48SL79u17oa56\nBj8/PzZt2sTJkyeJjIzkyZMn6mqWoKowWaNGDaKiovjll1/48MMPOXbsGKC6H8Xd3Z01a9YQFxdH\nnz59eOONN9SvtY+PD2fOnOHSpUvqrlAZTVqKnCY1Coriq7C1ZUI9homgei7i4bbtQkQGCuFlJUTQ\n3gLtsyhFRESIhQsXqgt9xcbGFmg/+amzUlaUldoyn3/+uejfv3+W59555x11Ea64uDgxevRoUaVK\nFVG1alUxZ84ckZqaql7X19dXuLi4CAsLC1G/fn1x9uzZF47Rvn17AQgzMzNhbm4utm3bJo4dOyYc\nHByEj4+PsLOzEx4eHuLhw4eiV69eomLFitkWKevYsaP45ptvhBBCfPvtt6Jt27bivffeEzY2NsLZ\n2Vns378/13OdN2+eaNOmjZg+fbro1atXlmWZ953h2LFjwt7eXgghxPXr14WBgYH466+/8npJ89S6\ndWuxdu1a9eN169aJVq1aZbtu//79xaJFi9SPT548KcqVKycePXokEhMTBSCio6PVy8eOHSs8PDyE\nEEKsWLFC9OzZU70sLS1NmJiYiCNHjgghhHjllVfEjh071Mu///57Ua1atRJTW6bEMmvRAttXHWHv\nO7oO5QVCCL755htcXV3x9vZW13yWM2H0z9tvv83+/fvVb/XT0tLYsWOHuqb4iBEjMDIyIjg4mN9/\n/51ff/1VPXzxww8/4O3tjZ+fHwkJCezZs0fdJSiz48ePA6p65ElJSepywvfu3ePhw4eEhYXh6+tL\neno6o0aNIiwsjNu3b2NqapprX9S//vqLevXq8eDBA2bOnImnp2eunxH5+fnh7u6Ou7s7hw4dIioq\nSuPX6ejRo1SrVo2WLVvmuI6Pj0+OrfUy/9+6fPmyulQv5N5aLyMZZn6cnJzMjRs31M8/v/zSpUs5\nbpvX8oiIiCylmItK6Z4tk5oMp9dB5Hmo0w3sC36rrjbdvHmTsWPHcuzYMdzc3NTNj6Xis/DvhVx9\neLVIj+FS3oUPWn6Q53pOTk68/PLL/PzzzwwfPpzffvsNMzMzXn31VaKiojhw4ABxcXGYmppSqVIl\npk+fjq+vL+PHj2fdunXMnDmTFi1aAOT798jAwIB58+ZRrlw5AHWT6Axz5syhU6dOucY+duxYQPVH\naNKkSURFRVGlSpUX1v39998JCwtj0KBBVKxYkVq1arFlyxamT5+uUayatNabNWsWs2bNynNfz7fX\ns7a2JikpST31OLMePXqwaNEiBg0ahK2tLQsXLgTg8ePHWFpa0rZtWz799FO++OILgoKC+PHHH9V1\n9Lt06cKsWbPw9/enTZs2LFy4kGfPnvH48WP1vpctW0anTp1IS0tj+fLlgKpbVVEr1Vfu3L8Kl3ZC\nOWtw3wHWDrqOiNTUVDp37syZM2dYu3YtR48elYldYujQoWzduhWALVu2qK/aw8LCSElJoWrVqtjY\n2ODo6Mj48eOJjlbd6B0eHk6tWrUKfNxKlSphYmKifvz48WPGjx+Pk5MTVlZWdOjQgbi4ONLS0rLd\nPnMSNzMzA1SJMzsbN26ka9euVKxYUX3OmWfeGBkZFUtrPXixvV5CQgIWFhbZTl4YPXo0b7/9Nm5u\nbjRo0ED9xy6j1+z333/PrVu3cHR0ZOLEibi7u6uXubi4sHHjRqZMmULVqlV58OABrq6u6uVz5syh\nWbNmNG3alDZt2tC3b1+MjY1zbLKiTaX7yl2kg1M7eMtX15Fw7do1atWqhZGRERs3bqRWrVrqf2Cp\n+GlyRV2cBg4cyHvvvUdERAS7du3i1KlTgKr1Xbly5Xjw4AFGRkYvFJRydHTMsY2bJp5PZosXL+ba\ntWv89ddfVKlShQsXLtCsWbNCT8d98uQJO3bsIC0tTf0HITk5mbi4OAIDA2nSpAnVq1cnNDQ0y3YZ\nSRNUzagnT57MmTNnaN68ebbHWbBgAQsWLMgxjow/PA0aNCAwMFA9xBMYGEiDBg2y3Sbj3c28efMA\n+PXXX3FwcFC3OXRycmLfvn3q9YcOHZpl6GjAgAEMGDAAULUT3LBhg/qdlqmpKV9//bW6TpSvry+v\nvPKKustVUSrdV+4ApjY6vWJ/9uwZ8+bNo1GjRqxcuRJQtUWTiV3KrFKlSri5uTFq1Chq1Kih7h1a\ntWpVunbtynvvvUdCQgLp6encvHmTgIAAAMaMGcOXX37J2bNnEUIQHBycY99QTdrhJSYmYmpqio2N\nDQ8fPlQntML6+eefMTQ0JCgoSN1a78qVK7Rv3x4/Pz8ABg8ezLfffsvff/+NEILr16+zZMkS9TBR\nnTp1mDRpEm+//Tb+/v48e/aMp0+fsm3bNnx8VD1xP/zwwxxb62V+RzF8+HC++uor7ty5Q2RkJIsX\nL2bkyJHZxv7w4UNu3ryJEIKgoCBmzJjBxx9/rO7deuXKFRITE3n27BmbN2/m119/ZcaMGertz549\nS1paGvfv32f8+PH06dNHPb0z4/hCCP78808+/fRTrb3medLkU9ei+NLGbJnQ1xoKsXVogfajDX/9\n9Zdo2LChAMTQoUPF/fv3i+xYcrZM7krybJkMfn5+AsgyM0MI1WyZCRMmCAcHB2FlZSWaNm0qtm7d\nql6+evVqUbduXWFubi4aNGggzp07l+3+V69eLapUqSKsra3F9u3b1bNlMrtz547o2LGjMDc3F3Xq\n1BFr1qwRgLqlXnazZTIDxI0bN144drdu3cSMGTNeeH779u3Czs5Ovf/169cLV1dXYWlpKWrVqiU+\n//xzERcXp14/PT1dLF26VLi6ugpTU1Nhb28vBg0aJC5dupTj65qd9PR08b///U/Y2toKW1tb8b//\n/U89Y00IIczNzcXx48eFEEJcu3ZN1K1bV5iamorq1auLxYsXZ9nXkiVLRMWKFYWZmZlo27atOH36\ndJblbdu2FRYWFsLW1laMGzdOJCUlqZcFBAQIJycnYWpqKurWratuq1ccs2Vkci+gJUuWCAMDA+Hg\n4CD27i36KZgyueeuNCR3TciWc/pBToUsgcS/Y5MtW7Zk7NixXL58md69e+s4KkmSpKxK9weqxSg+\nPp6ZM2diamrK0qVLadOmDW3atNF1WJIkSdmSV+4a2Lt3L66urqxbt45y5coVemaBJElSUZPJPRf3\n799n6NChvPHGG1SoUIE///yThQsXykJfkiSVeDK55yI+Pp79+/czb948zpw5o567KkmSVNKVyjF3\nQ5EG4X+pGmMbaPdmgPDwcDZv3sysWbOoXbs2YWFhWW5jliRJKg1K5ZW7IWmqxG5WAdq/p5V9pqen\ns2bNGho0aMD8+fPVdwXKxC5JUmlUKpO7mnkFqNok7/XycOPGDV577TUmTpxIy5Yt+eeff2Q9GEmS\nSrVSOSyjTampqXTp0oW4uDjWr1/PqFGj5AemkiSVeqX7yr0Qrly5QmpqKkZGRmzatImgoCBGjx4t\nE7tUammjzR6At7c3Hh4e+d5OCEHNmjVxdXV9YZmzszNHjhzJ8tx3331Hu3bt1I+fPXuGt7c3derU\nwdzcHGdnZ0aPHv1CsbG8JCcnM3r0aKysrKhSpQpfffVVruvm1AYvOTkZT09PnJycsLS0pFmzZhw4\ncCDL9uvWraN27dpYWFjQvXt3IiMjXzjGs2fPcHFxKfZ6U3qX3JOTk/Hy8qJx48bqSm3t27fH3t5e\nxyFDs2YAAA18SURBVJFJUul2/PhxoqOjCQkJ4XQBWmAOGDCAPXv2sGXLFuLj4wkMDOSVV17h6NGj\n+dqPt7c3N27cICwsjGPHjrFo0SIOHjyY7bq5tcFLTU3F0dGRgIAA4uPj+fTTTxk0aJD6j01AQAAf\nfvghu3fv5uHDh9SoUYO33377hWN88cUXVK5cOX8vhjZoUqOgKL4KU1vm1bXdRWi7GiK0X7d8bXvq\n1Cnh6uoqADFs2DDx4MGDAsWgC7K2TO5Kcm0ZXbXZE0KIvXv3iiZNmghra2vRunVrERgYqN7Gx8dH\n2NvbCwsLC1G3bl1x5MgRceDAAWFsbCyMjIyEubm5aNy4scbnOWrUKDF06FDRr18/MXny5CzLnJyc\nxOHDh7M8l1GYLCEhQRw+fFiYmJiI27dva3y8nNjb24tDhw6pH8+dO1cMHjw423VzaoOXk0aNGomd\nO3cKIYR47733xKRJk9TL7ty5IwARHBysfi4kJES4uLiI/fv3ZyniJmvLZKPjhVhm7rjL0zjjfG23\nePFi2rRpQ2JiIvv378fPzy/bdmWSpG26arN37tw5Ro8ezdq1a4mJiWH8+PG88cYbJCcnc+3aNb7+\n+mtOnz5NYmIihw4dwtnZme7du/Phhx8yePBgkpKSCAwM1OgcHz9+zM6dO9Ut9rZt28azZ880fo2O\nHDlCy5Yt1bXdszNp0qQc2+s1btwYUDWhj4yMLFSLvZza4EVFRXH9+nV1XfjstgXULfYApk6dyoIF\nCzA1NdXkZdCqUveBauugeKrff4aJTQpWbRvluX56ejoGBga0bt2aCRMm4OPjg5WVVTFEKunSvQUL\nSL5StG32ytV3ocqHH+a5nq7a7H3zzTeMHz+eVq1aAao/IgsWLODPP//EwcGB5ORkgoKCqFSpEs7O\nzgV6DTL89NNPlCtXjq5du5KWlkZqaiq//PIL/fr102h7TVrsrVq1ilWrVuW6TkZN9+db7CUmJma7\nfk5t8B4/fpxlHykpKbi7uzNixAh1rfaePXsyePBgJkyYQJ06dfjkk09QFEXdYm/Xrl2kpqbSr18/\n/P39c38BikCpu3I3oBx3Klrg1DkG29ez79YCqo4onp6eTJs2DYA2bdqwatUqmdglndBFm72wsDAW\nL16c5Qo3PDycyMhIateuzdKlS/H29qZy5coMGTIk2w8DNbVx40YGDRqEkZER5cqV46233tJJiz0L\nCwuAF1rsZe5ulVlObfAyj5Gnp6czbNgwXnrpJfXndKDqHDVv3jz69++Pk5MTzs7OWFpaUq1aNR49\nesTMmTNZsWJFoc+pwDQZuymKr4KOue/v/KY42KGLEF5WQlzYlu06u3btElWrVhWGhoZi9uzZWYr0\nl1ZyzD13JXnMXQghoqOjhYmJiQgPDxfW1tbqeCMjI4WJiYm6mcXzY7Fdu3YVS5cu1egYPNdIY9y4\ncWL+/Pl5bhcfHy+GDBkiPDw8hBBCeHt7C3d3d42OKYQQ4eHhwsDAQFhZWQk7OzthZ2cnLC0thbGx\nsbqBTadOncSqVauybDf7/+3df2zU9R3H8ecbQZrWSgmsR7MOQStK1YSaOiEs0EWyMI34D2yamBXS\nTHQ//MN/hBjJ4hTCks2ExAlEmNmSbbCZbITYjIxZXQx1bYaIWBmdtKO1QUmhhbRXSu+1P+6obWnv\nvm3vR+/6fiSX3I/Pfe/9vu/du9fP5/v9fLZuVXV19bA+93Pnzo35Ops3b1ZBQcGol/Ly8sF2JSUl\nOnLkyODtF198ccw+95H27Nmj5cuXD96ORCLauHGjqqqq1NPTE/e5p0+fVn5+vjo7O3X8+HHNnDlz\n8P2YO3euZsyYoVAopLNnz/piHaOJV9zPnz+vDRs2CNCyZctGHXjKVl7c45vqxV2S1q5dqzVr1mjZ\nsmXD7l+3bp2effZZdXV16dKlS2publZdXZ0k6eDBgyotLVVjY6MikYjOnDmjlpaWUbcfCoWGDSQ2\nNDSotLRU9fX1ikQiunLlig4fPqzu7m59+umnOnr0qMLhsPr6+rRp0yZVV1dLiq7otHLlSg0MDATK\na/v27br77rvV0dEx7LJ48WLt2rVLkrR7924tWbJETU1NikQiamhoUCgUUm1t7WChe/TRR1VZWanG\nxkb19/eru7tbr7/+uvbt2zeu9/n555/XqlWr1NnZqaamJi1YsEC1tbWjtm1ra1N7e7sikYiOHTum\n0tLSYe/h5s2b9eCDD+ry5cs3PLe3t1cnT55UJBJRa2urVq9era1bt0qS+vv7h70Xb731lkpKStTR\n0aFr1655cR9NvOJ+5swZFRUV6ZVXXtHVq1cntP2pyot7fNlQ3NO9zJ4k1dbWqrKyUnPmzNGCBQu0\nfv16dXd368SJE3rggQcGl4d75JFH1N7eLkm6cOGCVq5cqaKiIlVUVCTM66677hos4kPt3LlT17/n\nAwMD2rFjh8rKylRYWKilS5fqjTfekPTVfyt9fX3atm2b7rjjDuXn52vhwoWqqalRa2trwhiGCofD\n2rRpkwoLC1VcXDxs2bzW1lYVFBQMbnOsZfAkqaWlRYBmz5497L+E620uXryo++67T/n5+QqFQtqy\nZcuwo5yGGrnk4ZQp7sBa4DTQDGwZ5fHZwIHY4x8AixJtM1nFvbW1VS+//PJg10uuLtnlxT2+bCju\nQeTq5zcez3l0KT8U0sxuAl4DvguUA0+Y2chT0GqAi5LKgFeBnZMaCIij4OYZFM3oISLx6wNHuOee\ne9i+ffvgRF9jDZw459x0EuRomW8CzZI+k3QV+CPw2Ig2jwHXh8b/DDxkKTqPf0lZHv3WTNWbPfx4\nx35WrFjBqVOnfKIv55wbIkhx/zpwbsjttth9o7aRdA3oAlJyhtD873+LJ//9BSe78vnN/v2DJ184\n55z7SpCTmEb7BT5yEdEgbTCzp4CnAEKh0AQP7C/nma07KCsrY968ebz77rsT2Eb2uXLlSkZOhMik\n8eQc70SVbDIwMJATeYyH5zy6cDg8qe98kOLeBgw9J7gUGHm2w/U2bWY2E5gDdI7ckKS9wF6AyspK\nVVVVTSDkqMk8NxvV1dV5znE0NTVxyy23ZP2snpcvX55240ae840kkZeXR0VFxYRfI0i3TANwp5kt\nNrObgceBQyPaHAKqY9fXA/+Ijeo6lxazZs2it7c302E4lxS9vb2DZ+9OVMLiHutD/wnwN6AJOCjp\nlJm9ZGbrYs32AfPMrBl4DtgyqaicG6fi4mLa29vp6enBf1e4bCWJnp4e2tvbJz1NcKCJwyS9Dbw9\n4r5tQ66HgQ2TisS5Sbg+Z9Dnn39+wxwm2SQcDpOXl5fpMNLKcx5u1qxZhEKhSc+DlXWzQjo3lltv\nvTXrJ4arq6ubVD9rNvKcUyPrZoV0zjmXmBd355zLQV7cnXMuB3lxd865HGSZOmzMzL4EWif49PnA\nhSSGkw085+nBc54eJpPzbZK+lqhRxor7ZJhZo6Sx19jLQZ7z9OA5Tw/pyNm7ZZxzLgd5cXfOuRyU\nrcV9b6YDyADPeXrwnKeHlOeclX3uzjnn4svWX+7OOefimNLF3czWmtlpM2s2sxtmmjSz2WZ2IPb4\nB2a2KP1RJleAnJ8zs0/M7CMzO2pmt2UizmRKlPOQduvNTGaW9UdWBMnZzL4X29enzOz36Y4x2QJ8\nthea2Ttmdjz2+X44E3Emi5ntN7MvzOzjMR43M9sVez8+MrP7kxpAkFW0M3EBbgL+C9wO3AycAMpH\ntPkRsDt2/XHgQKbjTkPO3wbyY9efmQ45x9oVAu8B9UBlpuNOw36+EzgOzI3dLs503GnIeS/wTOx6\nOdCS6bgnmfMq4H7g4zEefxioJbqS3XLgg2S+/lT+5T6lFuZOk4Q5S3pHUk/sZj3RlbGyWZD9DPBz\n4BdAOJ3BpUiQnH8IvCbpIoCkL9IcY7IFyVnA9Wk953Djim9ZRdJ7jLIi3RCPAb9VVD1QZGYlyXr9\nqVzcp9TC3GkSJOehaoj+5c9mCXM2swrgG5IOpzOwFAqyn5cAS8zsfTOrN7O1aYsuNYLk/DPgSTNr\nI7p+xE/TE1rGjPf7Pi5TeT73pC3MnUUC52NmTwKVwOqURpR6cXM2sxnAq8DGdAWUBkH280yiXTNV\nRP87+6eZ3SvpUopjS5UgOT8BvCnpl2a2AvhdLOdI6sPLiJTWr6n8y308C3MTb2HuLBIkZ8xsDfAC\nsE5SX5piS5VEORcC9wJ1ZtZCtG/yUJYPqgb9bP9VUr+ks8BposU+WwXJuQY4CCDpGJBHdA6WXBXo\n+z5RU7m4T8eFuRPmHOui2EO0sGd7PywkyFlSl6T5khZJWkR0nGGdpMbMhJsUQT7bfyE6eI6ZzSfa\nTfNZWqNMriA5/w94CMDMlhIt7l+mNcr0OgT8IHbUzHKgS1JH0rae6RHlBKPNDwP/ITrK/kLsvpeI\nfrkhuvP/BDQD/wJuz3TMacj578B54MPY5VCmY051ziPa1pHlR8sE3M8G/Ar4BDgJPJ7pmNOQcznw\nPtEjaT4EvpPpmCeZ7x+ADqCf6K/0GuBp4Okh+/i12PtxMtmfaz9D1TnnctBU7pZxzjk3QV7cnXMu\nB3lxd865HOTF3TnncpAXd+ecy0Fe3J1zLgd5cXfOuRzkxd0553LQ/wGMz66r/8+kUQAAAABJRU5E\nrkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fc2d6dba550>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "wv_model = LogisticRegression().fit(X_train_wv, y_train)\n",
    "\n",
    "for name, X, y, model in [\n",
    "    ('bow train', X_train_bow, y_train, bow_model),\n",
    "    ('bow test ', X_test_bow, y_test, bow_model),\n",
    "    ('vec train', X_train_wv, y_train, wv_model),\n",
    "    ('vec test ', X_test_wv, y_test, wv_model)\n",
    "]:\n",
    "    proba = model.predict_proba(X)[:, 1]\n",
    "    auc = roc_auc_score(y, proba)\n",
    "    plt.plot(*roc_curve(y, proba)[:2], label='%s AUC=%.4f' % (name, auc))\n",
    "\n",
    "plt.plot([0, 1], [0, 1], '--', color='black',)\n",
    "plt.legend(fontsize='large')\n",
    "plt.grid()\n",
    "\n",
    "assert roc_auc_score(y_test, wv_model.predict_proba(X_test_wv)[:, 1]) > 0.92, \"something's wrong with your features\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If everything went right, you've just managed to reduce misclassification rate by a factor of two.\n",
    "This trick is very useful when you're dealing with small datasets. However, if you have hundreds of thousands of samples, there's a whole different range of methods for that. We'll get there in the second part."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}