{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false, "slideshow": { "slide_type": "skip" } }, "outputs": [], "source": [ "%matplotlib inline\n", "\n", "from __future__ import print_function, division\n", "\n", "from collections import defaultdict\n", "import matplotlib\n", "import matplotlib.pyplot as plt\n", "from mpl_toolkits.mplot3d import axes3d\n", "import numpy\n", "import scipy\n", "import scipy.spatial\n", "import pandas\n", "import gensim\n", "import re" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false, "slideshow": { "slide_type": "skip" } }, "outputs": [], "source": [ "def plot_vecs(corpus, query=None, labels=None):\n", " fig = plt.figure(figsize=(4, 4))\n", " c = numpy.array(corpus)\n", " if len(c[0]) == 2:\n", " ax = plt.gca()\n", " ax.quiver([0]*len(c),[0]*len(c),c[:,0],c[:,1],angles='xy',scale_units='xy',scale=1)\n", "\n", " if query:\n", " q = numpy.asarray(query)\n", " ax.quiver([0],[0],q[0],q[1],angles='xy',scale_units='xy',scale=1, color='r')\n", "\n", " ax.set_xlim([-0.1,c.max()])\n", " ax.set_ylim([-0.1,c.max()])\n", " else:\n", " ax = plt.gca(projection='3d')\n", " for q in c:\n", " ax.quiver(0, 0, 0, q[0], q[1], q[2], length=numpy.linalg.norm(q), arrow_length_ratio=0.1, pivot='tail', color='black')\n", "\n", " if query:\n", " q = numpy.asarray(query)\n", " ax.quiver(0, 0, 0, q[0], q[1], q[2], length=numpy.linalg.norm(q), arrow_length_ratio=0.1, pivot='tail', color='r')\n", "\n", " ax.set_xlim([0, c.max()])\n", " ax.set_ylim([0, c.max()])\n", " ax.set_zlim([0, c.max()])\n", " lang=labels\n", " if lang:\n", " if len(lang) >= 3:\n", " ax.set_zlabel(lang[2], fontsize=20)\n", " if len(lang) >= 2:\n", " ax.set_ylabel(lang[1], fontsize=20)\n", " if len(lang) >= 1:\n", " ax.set_xlabel(lang[0], fontsize=20)\n", " fig.tight_layout()\n", " plt.show()" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Building Search Engines with Gensim\n", "\n", "\n", "![tripy](./images/tripython-banner.png)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- PhD candidate @ University of Alabama" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Roll tide ππππππ―" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Search applications for software maintenance tasks\n", " - Code search\n", " - Change request triage" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Currently visiting researcher @ ABB\n", " - Looking for new adventures in 2016!" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "notes" } }, "source": [ "- Searching through large code bases using more than keyword search\n", "- Searching for developers most apt to handle a bug/issue report" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "- I have a Twitter βοΈproblemβοΈ\n", " - I tweet really dumb things sometimes" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- I am looking for a job" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "notes" } }, "source": [ "My advisor says I have a Twitter problem. I love Twitter and treat it seriously and facetiously at the same time. Also, jobs.\n", "\n", "These things don't mix well, apparently.\n", "\n", "Admitted to being intoxicated and breaking law. Whoops.\n", "\n", "Do employers want to hire someone that drinks at noon?" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "
I was apparently spotted on the local news being a drunk Alabama fan that flooded the streets after the win last night. Whoops.
— christop corley (@excsc) January 10, 2012
\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"instead of lunch I am just going to have a bunch of beers and eat pretzels
— christop corley (@excsc) September 29, 2014
\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"**CLARIFICATION: I do not \"have a drinking problem\", as my mother likes to claim. I am probably actually quite sober right now!* "
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Let's make a search engine"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"source": [
"So I have a lot of tweets, lets make a search engine to find all of these bad tweets for deletion\n",
"\n",
"Lots of different methods for making search engines, but mainly two essential behaviors we want"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"`search(\"drunk\") => [tweets containing the word 'drunk']`"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"`search(\"drunk\") => [tweets *related* to the word 'drunk']`"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [],
"source": [
"import gensim"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"I'm a `gensim` contributor\n",
"\n",
"`gensim`'s βοΈdesignβοΈ is not always great, certainly better tools exist that can supplement or replace parts of this talk, e.g.:\n",
"\n",
"1. `whoosh`\n",
"2. `nltk`\n",
"3. `scikit-learn`"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"source": [
"OMG WHAT ABOUT GREP?"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"### `grep` is great, but\n",
"\n",
"- must search through all documents\n",
"- not aware of language-specific semantics\n",
"- does not rank by relevence"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"source": [
"1. First relates to the efficiency (speed): we want results fast!\n",
"2. Second relates to the effectiveness (quality): we want to avoid false-positives\n",
"3. Third relates to the effectiveness (quality): we want things ordered by relevence"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Basics\n",
"\n",
"1. **Prepare** the corpus\n",
"2. **Model** the corpus \n",
"3. **Index** the corpus\n",
"4. **Query** fun-time!"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"source": [
"These are the basic steps, all of which gensim can help. However, gensim's most powerful features fall out of number 2!\n",
"\n",
"Gensim has some powerful models that can help with many tasks requiring measuring document similarity, not just search.\n",
"\n",
"The model we choose will influence how we prepare and index the corpus."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [],
"source": [
"cat_phrases = [\"meow\",\n",
" \"meow meow meow hiss\",\n",
" \"meow hiss hiss\",\n",
" \"hiss\"\n",
" ]"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"source": [
"Let's pretend we have 4 cats that all have different signature catch phrases.\n",
"\n",
"Now, we need a good way to represent these cats, but in a way a computer can understand easily."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"cat_lang = [\"meow\", \"hiss\"]\n",
"\n",
"cat_phrase_vectors = [[1, 0],\n",
" [3, 1],\n",
" [1, 2],\n",
" [0, 1]]"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "-"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAARkAAAEbCAYAAADu2PAcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4FGW69/HvTVhliSICCrK6oKKCMsqi0OIgBARFFuke\nx3Gb4QyijguKOO/InHcA9agDgoqMu54EFZVFERiQyKIiIBFkM8geFmVIhCRAtvv80U3bhASS0NXV\n6b4/19VXuqueVN0pkh9P1dP1tKgqxhjjlCpuF2CMiW0WMsYYR1nIGGMcZSFjjHGUhYwxxlEWMsYY\nR7keMiJSQ0SWicgqEVkrImNLafeCiKSLSJqItIt0ncaYiqnqdgGqekRErlPVXBFJAJaKSBdVXXq0\njYgkAa1V9XwRuRqYDHR0q2ZjTNm53pMBUNXcwNMa+GvKLNbkJuDtQNtlQKKINIpchcaYioqKkBGR\nKiKyCtgDpKrqumJNmgA7Ql5nBJYZY6JcVISMqhapanugKdBVRLq5XZMxJjxcvyYTSlUPiMinQAfg\ni5BVGcC5Ia+bBpYdQ0TsRixjHKKqUpHvc70nIyINRCQx8LwW0ANIK9ZsJnB7oE1HIEtV95a0PVUt\n8fHkk0+Wui7SD6sleuuwWkp+nIpo6MmcDbwlIoI/9N5R1QUiMhRQVZ2iqrNFpLeIbAJygDvdLNgY\nU3auh4yqrgGuKGH5K8VeD49YUcaYsHH9dClSPB6P2yUEWS3Hi5Y6wGoJNznV861oIiIaSz+PMdFC\nRNDKeuHXGBPbLGSMMY6ykDHGOMpCxhjjKAsZY4yjLGSMMY6ykDHGOMpCxhjjKAsZY4yjLGSMMY6y\nkDHGOMpCxhjjKAsZY4yjLGSMMY6ykDHGOMpCxhjjKAsZY4yjLGSMMY6ykDHGOMpCxhjjKAsZY4yj\nLGSMMY6ykDHGOMpCxhjjKAsZY4yjLGSMMY5yPWREpKmIfC4ia0VkjYjcX0KbbiKSJSLfBh5/daNW\nY0z5VXW7AKAAeEhV00SkDrBSROap6oZi7Rapaj8X6jPGnALXezKqukdV0wLPs4H1QJMSmlbow76N\nMe5yPWRCiUgLoB2wrITVnUQkTUQ+FZGLI1qYMabCouF0CYDAqdI04IFAjybUSqCZquaKSBIwHbig\npO2MHj06+Nzj8eDxeByp15hYlpqaSmpqali2Jaoalg2dUhEiVYFPgM9UdUIZ2m8BrlTV/cWWazT8\nPMbEGhFBVSt0ySJaTpdeB9aVFjAi0ijk+VX4w3F/SW2NMdHF9dMlEekC/A5YIyKrAAVGAc0BVdUp\nwEAR+TOQDxwCbnWrXmNM+UTF6VK42OmSMc6IhdMlY0yMspAxxjjKQsYY4ygLGWOMoyxkjDGOspAx\nxjjKQsYY4ygLGWOMoyxkjDGOspAxxjjKQsYY4ygLGWOMoyxkjDGOspAxxjjKQsYY4ygLGWOMoyxk\njDGOspAxxjjKQsYY4ygLGWOMoyxkjDGOspAxxjjKQsYY4ygLGWOMoyxkjDGOspAxxjjKQsYY4yjX\nQ0ZEmorI5yKyVkTWiMj9pbR7QUTSRSRNRNpFuk5jTMVUdbsAoAB4SFXTRKQOsFJE5qnqhqMNRCQJ\naK2q54vI1cBkoKNL9RpjysH1noyq7lHVtMDzbGA90KRYs5uAtwNtlgGJItIoooUaYyrE9ZAJJSIt\ngHbAsmKrmgA7Ql5ncHwQGWOiUNSETOBUaRrwQKBHYxyWl5fndgkmDkTDNRlEpCr+gHlHVWeU0CQD\nODfkddPAsuOMHj06+Nzj8eDxeMJWZyxZv349S5Ys4Y9//KPbpZgolJqaSmpqali2Jaoalg2dUhEi\nbwP7VPWhUtb3Bu5V1T4i0hEYr6rHXfgVEY2Gnyfa7dy5k86dOzNhwgT69+/vdjmmEhARVFUq8r2u\n92REpAvwO2CNiKwCFBgFNAdUVaeo6mwR6S0im4Ac4E73Kq7cMjMz6dWrFzt27ODCCy90uxwTB6Ki\nJxMu1pM5sUOHDtGjRw+WLl1KQkICOTk51KhRw+2yTCVwKj2ZqLnwa5xVUFDAkCFDWLp0KQAtW7a0\ngDERYSETB1SV//qv/2LmzJnBZW3atHGxIhNPLGTixJQpU3jiiScASExMtOsxJmIsZOKAiHDw4EEm\nTZpEhw4dmD9/PpdffrnbZZk44frokomMl19+mV9++YVRo0bRoUMHrrjiCrdLMnHCRpfiQG5uLi1a\ntKBBgwZ8//33VKliHVhTPpX6fTLGea+//jo///wzzz33nAWMiTjrycS4vLw8zjvvPBISEvjhhx+o\nVq2a2yWZSsh6MqZUycnJ7Nixg5deeskCxrjCejIxrLCwkEsuuYSsrCy2bt1KzZo13S7JVFL2jl9T\noo8//piNGzfy8MMPW8AY11hPJkapKldeeSVbtmxh+/bt1K1b1+2STCVmPRlznLlz57Jq1Sruv/9+\nCxjjKuvJxKiuXbvy7bffsm3bNs4880y3yzGVnPVkzDEWL17M4sWLGTp0qAWMcZ31ZGJQ7969WbBg\nAZs3b6ZJE5tv3Zw668mYoFWrVvHZZ59xxx13WMCYqGAhE2PGjRtHlSpVePTRR90uxRjAQiambNy4\nkWnTpjFkyBBat27tdjnGABYyMeXpp59GVRk5cqTbpRgTZBd+Y8T27dtp3bo1vXv3ZsaMkj66ypiK\ni+iFXxFpLyLDRCQxZFltEXlLRLJEZJeIPFCRYkzFPfvssxQUFPD444+7XYoxxyh3T0ZEpgLXqmqT\nkGUvAMOBbKAG/ru7k1R1XhhrLUttcdmT+emnn2jevDmdO3dmwYIFbpdjYlCkh7A7AAtDdl4N+APw\nDdAQaAnsA+6vSEGm/MaPH8/hw4cZNWqU26UYc5yKhExDYGfI6w5AXeAVVT2sqruAGcBlYajPnERW\nVhYvvvgiV111Fd27d3e7HGOOU5GQUY6d7OqawLIvQpb9DJx1CnWZMnrppZc4cOAAjz/+OCIV6s0a\n46iKXJNZA/yiqtcEXi8BzlXV5iFt3gB6quo54Sy2DLXF1TWZ3NxcmjdvTsOGDVmzZo3N32scE+np\nN98H/i4i04DDQCdgfLE2FwE/VqQgU3avvvoq+/bt45///KcFjIlaFenJ1AHm4g8XgDTgOlX9JbC+\nJbAJGKeqfy3D9l4DbgT2qupx13FEpBv+azybA4s+UtV/lLKtuOnJ5OXl0bp1a6pWrUp6ejpVq9p0\nzcY5Ee3JqGo20EVE2gYWrVPVotAmwC3AijJu8g1gIvD2CdosUtV+5a01lr377rvs3LmTl19+2QLG\nRLWoeMeviDQHZp2gJ/OIqvYtw3bioidTWFjIRRddxMGDB9myZYvN32scF+l3/CaIyGklLO8uIhNE\nZGzglCmcOolImoh8KiIXh3nblc6HH35Ienq6TRBuKoWK9LOfBf4sIo1CrsMMAf4XOJp0fxSRK1R1\nRxhqXAk0U9VcEUkCpgMXlNZ49OjRwecejwePxxOGEqKHqjJ27FjOOOMMhg4d6nY5JkalpqaSmpoa\nlm1V5MLvSuAnVU0KWbYe/5v0HgAaA+OASar6YBm3WerpUglttwBXqur+EtbF/OnS7Nmz6dOnD08+\n+eQxgWqMkyJ9W8G5+EePju68FXAhMFFV31XVZ4HPgF7l2Kbway/o2BUijUKeX4U/GI8LmHigqowZ\nM4batWtz3333uV2OMWVSkdOlesCBkNdd8I8ozQlZtha4riwbE5FkwAOcKSLbgSeB6oCq6hRgoIj8\nGcgHDgG3VqDmmLB48WK+/PJLHn74YZsg3FQaFTld2gJ8paq+wOu3gAHAGaqaH1g2EbhNVc8Ic70n\nqy2mT5d69erFwoUL2bJlC+ecE9E3U5s4F+l3/H4N9BORG/G/43cgsOBowAS0BDIqUpAp2cqVK5k7\ndy5Dhw61gDGVSkV6MpcCy/DPGwNQBFyjqssC62sCe4Fpqnp3GGstS20x25MZOHAgH3/8Menp6bRq\n1crtckycifQ7fteIyNX455ABeE9Vl4c0aQ98DqRUpCBzvPXr1/PRRx/h8/ksYEylExXv+A2XWO3J\n3HHHHbz11lusWbOGtm3bnvwbjAmzU+nJWMhEua1bt3Leeedx4403Mn36dLfLMXHK0dMlEbk98PRj\nVT0Y8vqkVPVENz2aMnj22WcpLCy0CcJNpXXSnoyIFOF/H8xFqvpDyOsTfhv+97kkhKfMsom1nsye\nPXto0aIF11xzDfPnz3e7HBPHnL7wexf+UNkdeH1nRXZkym/8+PEcOXLEJgg3lZpdk4lSWVlZNGvW\njIsvvpivvvrK5u81ror0vUsmAl588UUOHjzIqFGjLGBMpVahnkxgIqkRwFXAGZQcVqqqEZ2yLVZ6\nMjk5ObRo0YLGjRvz3Xff2fy9xnURfTOeiPTBP6dLArAd2AgUVGTnpmRHJwifMGGCBYyp9CpyW8Fy\n4BLg5kh/DO3JxEJPJi8vj1atWlGjRg02btxo8/eaqBDpGyTbAlOjLWBixTvvvENGRgavvPKKBYyJ\nCRXpyfwMvK2qDztTUsVV9p5MYWEhbdq0IScnhy1btlCjRo2Tf5MxERDpnswCfv3MJRNG06ZNY9Om\nTTz33HMWMCZmVKQn0xz4Bv9nJY2Jpq5DZe7JqCrt2rVj586dbNu2jTp16rhdkjFBTt+79HoJi9cC\nfwfuEpE0IKuENhrp+WQqs9mzZ7N69Wr+/ve/W8CYmFLWe5cqwu5dKiNVpUuXLqxZs4Zt27ZRv359\nt0sy5hhOX5MJ9we1mWIWLVrEV199xYgRIyxgTMyxe5eiQM+ePfniiy/YsmULZ599ttvlGHMcu3ep\nEluxYgXz5s3jrrvusoAxMcl6Mi4bMGAAM2bMID09nZYt7czURCfryVRS69atC04QbgFjYpWFjIue\nfvppAEaOHOlyJcY4x06XXHJ0gvB+/frx0UcfuV2OMSdkp0uV0P/8z//YBOEmLrgeMiLymojsFZHV\nJ2jzgoiki0iaiLSLZH1O2LNnD6+99ho9evTgN7/5jdvlGOMo10MGeAPoWdpKEUkCWqvq+cBQYHKk\nCnPKP//5T5sg3MSNqLgmE7jpcpaqXlbCusnAQlV9L/B6PeBR1b0ltI36azKZmZk0a9aMtm3b8uWX\nX9r8vaZSiPVrMk2AHSGvMwLLKqVJkyaRnZ1tE4SbuBE3U6/l5uZy2mmnuVpDdnY248eP59JLL6VP\nnz6u1mJMqKKiInJzc8nOziYnJ+e4r6eiMoRMBnBuyOumgWUlGj16dPC5x+PB4/Gwbds25s6dy5/+\n9CfHiiyLf/3rX+zfv59JkybZBOEmqjz//POMGDHiuOXVq1enb9++p7ZxVXX9AbQA1pSyrjfwaeB5\nR+DrE2xHSzJp0iTt06dPiesi5fDhw3rOOedo69atNT8/39VajAm1e/dunTBhgtavX1/xf1qsAtqx\nY0fdvHmzqqoG/rYq9Pftek9GRJIBD3CmiGwHngSq4/+hpqjqbBHpLSKbgBwq8DG5n3zyCampqa6e\nMr399tvs2rWLKVOm2AThxnVZWVl89NFHpKSk8Pnnn1NUVBScLE1EePzxxxk9ejTVqlU75X1FxehS\nuJQ0upSdnc2ZZ55JXl4eM2fOPPWuXwUUFBTQpk0bDh06xObNm23+XuOKQ4cO8cknn5CcnMzs2bPJ\ny8ujTp063Hzzzfh8PurVq8egQYN499136d69+zHfG+mJxCuVf//73+Tl5QEwa9YsV0Lmgw8+4Mcf\nf+T555+3gDERlZ+fz4IFC0hOTubjjz8mOzub6tWr07t3b7xeLzfeeGOwd5+ens7q1atp0KBBWGuI\n+Z7MXXfdxRtvvAHA2Wefzc6dOyN60bWoqIjLL7+c3bt3s3XrVpu/1ziuqKiIL7/8kpSUFN5//332\n7dtHlSpVuO666/D5fNxyyy2cfvrp5drmqfRkXL/oG84HxS78FhYWasOGDVVEghezli9fXsbLYeEx\nc+ZMBfS///u/I7pfE1+Kioo0LS1NH330UW3WrFnw9/3qq6/W8ePH665du05p+5zChd+Y7slkZGTw\n9ddfk5KSwpw5c1iyZAlZWVl4PJ6I1KOqdOrUibVr17J9+3bOOOOMiOzXxI8ff/yRlJQUkpOTWb9+\nPQAXXXQRPp8Pr9dL69atw7If68mU0pM5atCgQVq7du2yhnbYfP755wroo48+GvF9m9i1a9cuHT9+\nvF511VXBHkuzZs30scce07S0NC0qKgr7PqnMQ9ixbOzYsdSoUYMHH3zQ7VJMJZeZmRkccl64cCFF\nRUU0aNCAYcOG4fP56NSpU9S+wdNCxiHffPMN8+fPZ9iwYTRu3NjtckwllJubGxxy/uyzz4JDzrfd\ndhter5frr78+LO9jcZqFjEPGjRtHQkJCiW/VNqY0+fn5zJ8/n+TkZKZPnx4ccu7Tpw8+n48+ffpQ\nq1Ytt8ssFwsZB6xdu5bp06dz++2306JFC7fLMVGuqKiIpUuXBoec//Of/1ClShW6d++Oz+ejf//+\n5R5yjiYWMg546qmnEBGbINyUSlX57rvvSE5OZurUqezY4Z/NpGPHjvztb39j8ODBMXOabSETZps3\nbyYlJYX+/ftz0UUXuV2OiTKbNm0KDjlv2LABgIsvvpgxY8YwZMgQWrVq5XKF4WchE2Y2Qbgpbvfu\n3bz33nskJyezfPlyAJo3b87IkSPxer1ceumlMT2BmYVMGO3evZvXX3+dG264gQ4dOrhdjnFRZmYm\nH374YXDIWVU566yzuPfee4NDzrEcLKEsZMLo+eefJy8vzyYIj1O5ubnMmjUrOOScn59P3bp1uf32\n24NDzvE4zUf8/cQO2b9/Py+//DKdO3ema9eubpdjIiQ/P5958+aRkpLC9OnTycnJoUaNGvTt2xef\nz0fv3r0r3ZBzuFnIhMnEiRPJycmxCcLjQFFREUuWLCElJYUPPvggOOT829/+Fq/XS//+/UlMTHS7\nzKhhIRMGBw8eZMKECVx22WX07t3b7XKMA1SVtLS04JDzzp07AejUqRNer5fBgwfTqFEjl6uMThYy\nYTBlyhQyMzN5+eWXrRcTY9LT00lJSSElJSU45Ny2bVuGDRvGkCFDaNmypcsVRr+YnurhqMGDBzN7\n9myys7PDvs/Dhw/TqlUrateuzYYNG0hISAj7Pkxk7dq1KzjkvGLFCgBatGiB1+sNDjnHG5t+00Vv\nvfUWu3fv5tVXX7WAqcT2798fHHJOTU1FVWnYsCHDhw/H5/PRsWNH66VWkIXMKSgoKOCZZ56hadOm\n/P73v3e7HFNOOTk5wSHnOXPmkJ+fT7169fjDH/6A1+ule/fucTnkHG52BE/B+++/z+bNmxk/fjzV\nq1d3uxxTBnl5ecEh5xkzZgSHnPv16xcccq5Zs6bbZcYUC5kKKioqYty4cTRo0IB77rnH7XLMCRQV\nFbF48eLgkPP+/ftJSEgIDjnffPPNNuTsIAuZCvrkk0/4/vvv+cc//kHt2rXdLscUo6qsWrWK5ORk\n3nvvveCQc+fOnfH5fAwaNIiGDRu6XGV8sJCpAFVlzJgx1K1bl3vvvdftckyIH374IXiX8w8//ADA\npZdeyr333suQIUNsfh8XWMhUwMKFC/nmm28YOXJkpZ5MKFZkZGQEh5xXrlwJQMuWLRk1ahRer5e2\nbdu6XGF8s5CpgLFjx1KzZk3+8pe/uF1K3Nq/fz/Tpk0jJSWFL774AlWlUaNG3H///Xi9Xq6++mob\nco4SUREyItILGA9UAV5T1aeLre8GzAA2BxZ9pKr/iGyVfsuWLWPBggUMHz7c3kYeYTk5OcycOZPk\n5GTmzp1Lfn4+iYmJ3HHHHfh8Pjwejw05RyHX/0VEpAowCbge2AUsF5EZqrqhWNNFqtov4gUWM27c\nOKpWrcojjzzidilxIS8vj7lz5waHnHNzc6lZsyY33XQTPp+PpKQkG3KOcq6HDHAVkK6q2wBEZCpw\nE1A8ZFzv+37//ffMmDGDO+64g+bNm7tdTswqKipi0aJFwSHnzMxMEhIS6NGjR3DIuV69em6Xacoo\nGkKmCbAj5PVO/MFTXCcRSQMygBGqui4SxYU6OkH4Y489FuldxzxV5dtvvw0OOWdkZADQpUsXfD4f\nAwcOtCHnSioaQqYsVgLNVDVXRJKA6cAFkSzg6AThAwYMoE2bNpHcdUzbuHFjcMg5PT0dgMsuu4z7\n7ruPIUOGWI8xBkRDyGQAzUJeNw0sC1LV7JDnn4nISyJSX1X3F9/Y6NGjg889Hg8ejycsRT7zzDMU\nFRXZBOFhsHPnzuCQ87fffgtAq1ateOKJJ/B6vVxyySUuV2hSU1NJTU0Nz8Yq+iHa4XoACcAmoDlQ\nHUgDLirWplHI86uAraVsq8QPCx80aJDWrl37ZJ8pXqqMjAytXr269urVq8LbiHf79u3TyZMna7du\n3VREFNBGjRrpAw88oF9//bUjHxJvwifwt1Whv3HXezKqWigiw4F5/DqEvV5EhvpX6xRgoIj8GcgH\nDgG3RrLGoxOEWy+mfLKzs48Zci4oKCAxMZE777wzOORs02PEPpu06iT+85//0Lx5c9q1a8fixYvt\nDV4ncXTIOTk5mZkzZwaHnPv164fX6yUpKYkaNWq4XaYpJ5u0ykE2QfjJFRYWBoecp02bFhxyvuGG\nG/D5fNx0003UrVvX7TKNSyxkTuDgwYO88MILXH755SQlJbldTlRRVVauXBkcct61axcA11xzTXDI\n+ayzznK5ShMNLGRO4JVXXiEzM5PJkydbLyZgw4YNwSHnTZs2AdCuXTseeOABhgwZQrNmzU6yBRNv\n7JpMKQ4fPkzLli2pW7cu69evj+sLlDt27AgOOa9atQqA1q1bByfWvvjii12u0DjNrsk44M0332TP\nnj2MGTMmLgNm3759wbucFy1aBEDjxo35y1/+gtfr5Te/+Y317kyZWMiUoKCggKeffpqmTZty2223\nuV1OxGRnZzNjxgySk5OZN28eBQUFnH766dx99934fD66desWl4FrTo2FTAmmTp3K1q1bmTBhQsxP\nEH7kyJFjhpwPHTpErVq1GDBgAF6vl169etmQszklFjLFxMME4YWFhXzxxRfBIeesrCyqVq0aHHLu\n16+fDTmbsLGQKWbmzJmsW7eOMWPGcNppp7ldTtioKitWrAgOOe/evRuArl274vV6GThwIA0aNHC5\nShOLLGRCqCpjx46lXr16DBs2zO1ywmL9+vXBz3I+OuTcvn17HnzwQYYMGcK5557rcoUm1lnIhFiw\nYAHLly/n8ccfr9QThG/fvp2pU6eSkpJCWloaAOeddx5/+9vf8Hq9NlWFiSgLmRCVeYLwffv28cEH\nH5CSksLixYsBOPvss3nwwQfx+XxceeWVNuRsXGEhE/DVV1+xcOFC7rvvvkozA9vBgweDQ87//ve/\ng0PO99xzDz6fj65du9qQs3GdhUxAZZkg/MiRI8yZM4fk5GRmzZoVHHIeOHAgXq+Xnj172pCziSoW\nMsDq1auZNWsWd955Z1Tee1NYWEhqaiopKSl8+OGHwSHnnj17Boec69Sp43aZxpTIQobonCBcVVm+\nfHlwyHnPnj0AdOvWLTjkfOaZZ7pcpTEnF/chs2nTJt577z0GDhzIhRde6HY5rFu3Ljjk/OOPPwJw\nxRVX8Mgjj3DrrbfStGlTlys0pnziPmSiYYLwbdu2BYecv/vuOwDOP/98nnzySbxeb1SEnzEVFdch\nk5GRwZtvvklSUhLt27eP6L5//vnn4JDzkiVLADjnnHN46KGH8Pl8XHHFFTbkbGJCXIfMc889R35+\nPqNGjYrI/g4ePMj06dODQ86FhYWcccYZ/OlPf8Lr9XLttdfakLOJOXEbMvv27eOVV17h2muv5Zpr\nrnFsP0eOHOGzzz4LDjkfPnyY0047jcGDB+Pz+bjhhhti/k5vE9/iNmReeOEFcnNzHenFFBYWsnDh\nwuCQ8y+//ELVqlXp1atXcMi5du3aYd+vMdEoLkPmwIEDTJw4kfbt29OzZ8+wbFNV+eabb0hOTub9\n999nz549iEhwyHnAgAE25GziUlyGzOTJk8nKygrLx5ysXbs2OOS8efNmAK688kpGjBjBrbfeSpMm\nTcJRsjGVVtyFzKFDh3j++ee58MIL6d+/f4W2sXXr1uCQ8+rVqwG44IILGD16NF6vlwsuuCCcJRtT\nqcVdyLzxxhvs3buXp556qlwjOT/99FNwyHnp0qUANGnShIcffhifz0f79u1tyNmYEsTVR6JkZmZy\n/vnno6ps2rSJatWqnXB7Bw4cCA45z58/n8LCQurXr8+gQYOCQ85VqlRx6scxJmrYR6KUUUpKCtu2\nbWPixImlBszhw4eZPXs2KSkpfPLJJxw+fJjatWtz66234vP56NGjhw05G1MOUREyItILGA9UAV5T\n1adLaPMCkATkAHeoalp59zNu3DgaNmzI3XfffczygoKCY4acDxw4QLVq1UhKSsLr9dK3b18bcjam\nglwPGRGpAkwCrgd2ActFZIaqbghpkwS0VtXzReRqYDLQsTz7KSwsZMOGDYwbN45atWqhqixbtiw4\n5Lx3715EBI/Hg8/n45ZbbqF+/fph/EmNiU+uhwxwFZCuqtsARGQqcBOwIaTNTcDbAKq6TEQSRaSR\nqu4t607y8vJITEykW7duPPHEE6SkpLBlyxYAOnTowGOPPcbgwYNjash5xYoVvPTSS/Tt25cePXrY\nnDPGFdEQMk2AHSGvd+IPnhO1yQgsK1PI7N69m6KiIgA6d+4MQNOmTfnd735H165dg8GSlpYWnHg7\nFqgqc+bM4Y033qB69ep0796dG2+8kb59+0bl5FwmRqmqqw9gADAl5PVtwAvF2swCOoe8ng9cUcK2\ntCTPPPOMAvYIeXi9Xt25c2eJx8uY4gJ/WxX6G4+GnkwGEPrfatPAsuJtzj1JGwBGjx4dfO7xePB4\nPIwYMYLGjRtTq1atuBpyVlWGDx/Onj17qFq1Kt26dQv2ZFq3bu12eSaKpaamkpqaGpZtuf4+GRFJ\nADbiv/C7G/gG8Krq+pA2vYF7VbWPiHQExqvqcRd+S3ufTLxasWIFEyZMoG/fvvTs2ZPExES3SzKV\n1Km8T8Z1/CdyAAAHIElEQVT1kIHgEPYEfh3CfkpEhuLvok0JtJkE9MI/hH2nqn5bwnYsZIxxwKmE\nTFScO6jqHFW9UFXPV9WnAsteORowgdfDVfU8Vb28pIA5mXB1/cLBajletNQBVku4RUXIREI0/WNZ\nLceLljrAagm3uAkZY4w7LGSMMY6Kigu/4SIisfPDGBNlKvXokjEmdtnpkjHGURYyxhhHxVzIiEgv\nEdkgIj+IyGOltHlBRNJFJE1E2rlRh4h0E5EsEfk28PirE3UE9vWaiOwVkdUnaBOJY3LCOiJ8TJqK\nyOcislZE1ojI/aW0c/S4lKWOSB0XEakhIstEZFWgnrGltCvfManoTU/R+MAfmpuA5kA1IA1oU6xN\nEvBp4PnVwNcu1dENmBmh43IN0A5YXcp6x49JGeuI5DFpDLQLPK+D/9YWN35XylJHJI/LaYGvCcDX\nQJdTPSax1pMJzk2jqvnA0blpQh0zNw2QKCKNXKgDICIzj6vqEiDzBE0icUzKUgdE7pjs0cDsiqqa\nDazHP31IKMePSxnrgMgdl9zA0xr4/7Ms/u9V7mMSayFT0tw0xf/BSpubJtJ1AHQKdDk/FZGLw1xD\neUTimJRVxI+JiLTA38NaVmxVRI/LCeqACB0XEakiIquAPUCqqq4r1qTcxyQapnqIVyuBZqqaG5he\ndDoQ7x/YFPFjIiJ1gGnAA4GehCtOUkfEjouqFgHtRaQeME9EuqnqF6eyzVjryYR1bhon61DV7KNd\nU1X9DKgmIm5NKhyJY3JSkT4mIlIV/x/2O6o6o4QmETkuJ6vDjd8VVT0AfAp0KLaq3Mck1kJmOXCe\niDQXkerAEGBmsTYzgdsBAnPTZGk55goOVx2h57EichX+N0buD3Mdx+yS0s/rI3FMTlqHC8fkdWCd\nqk4oZX2kjssJ64jUcRGRBiKSGHheC+iBf9AiVLmPSUydLqlqoYgMB+bx69w060PnplHV2SLSW0Q2\nEZibxo06gIEi8mcgHzgE3BruOo4SkWTAA5wpItuBJ4HqRPCYlKUOIntMugC/A9YErkEoMAr/iGDE\njktZ6iByx+Vs4C0REfy/t++o6oJT/fux2wqMMY6KtdMlY0yUsZAxxjjKQsYY4ygLGWOMoyxkjDGO\nspAxxjjKQsYY4ygLGWOMoyxkjDGOspAxxjjKQsYAELiZs0hEXheRViIyTUT2icgBEZknIm0D7RqI\nyKsisktEDonIchHxlLC9BBEZJiJficgvIpITmDry3sC9MSXVMFhEFgWmmswVkdUiMjJwk2lou69E\n5EjgJr7Q5V8EfoZ/FVveJrD8zVM9Tqb8LGRMcS3xT5p0FvAGMBe4HlgoIhcG1rXHP9vfe8BlwGwR\naXp0A4GpCz4FJgGJwP8Cr+C/+3oi8GbxnQbmk50KXBhoPzGwaiwwJ7DNo+bjv7n32pDvr4V/OkgN\n1Bvq+sDy+eU5ECZMIjFvqD2i/4H/rt8ioBAYWWzdXwPrsoAXi627LbDuuZBlowPLxhO4CTewXIBX\nA/voG7K8Y6D9FuCskOVV8E8tcExNwHWB9k+HLLshsGxOoH3LkHUfB5Y1cfs4x+PD9QLsER2PkJD5\nMTQYAuvODaw7CNQutq4KkAcsCLwWYB/+iYyqlLCfxMAf/NSQZf8KLLu7hPbnAwXAppBlNYBcYEXI\nsmeAI/h7M0XAPSH17Ac2uH2M4/URU/PJmLBI08BfZ4hdga8/qGpO6ApVLRKRvfhnSAP/tJD1gR+A\n/1fC5RfBPyfKRSHL2ge+LizeWFXTRWQn0FJE6qrqQVU9IiJfAh4ROUNVM/H3br5R1WWBeq7H32u6\nEjgd/6mYcYGFjCnul+IL1D8JV4nrAgrwf/QLwJmBr+cDfzvBfmqHPE8MfN1dStvd+HtTp+PvTQEs\nwB8s14nI5/iD6h+BdZ/z63WZo9djFpygFuMgu/Brwu1oEH2sqgkneJxXwvc0LmWbZxdrB/4gEeC3\n+MOmCr8GyefAWSJyGb+GzHG9JBMZFjIm3Dbgv0DcUUQSyvg9qwJfPcVXiEhr/KdiW9Q/ufVRy4ED\n+EOkO/5rNF8H1i3AH0B9gM74P0zOybmCzQlYyJiwUtVC/MPP5wATRaRm8TYi0lhEQq/JvI4/FP4q\nIg1C2lUBnuPXUanQ/RQBi/Cflg0Clqj/g/RQ1a3AVuABoBb+no1xiV2TMU74//jfPzMU6Bu4ZpIB\nNMQfCl3wT5a9HkBVvxKRZ4ARwPciMg3/JNVJwCXAYuDZEvazALgR/3t6il9zWQDcjV2PcZ31ZEwo\nDTzKu47QdapaoKr98X90xgb8py0PAT3x90qewP+GO0K+ZyTgxT8q9XvgvpC2N6hqQQn7XBBSV/He\nytF1+fh7PMYl9mkFxhhHWU/GGOMoCxljjKMsZIwxjrKQMcY4ykLGGOMoCxljjKMsZIwxjrKQMcY4\nykLGGOMoCxljjKP+Dy1PFB26ov9TAAAAAElFTkSuQmCC\n",
"text/plain": [
"