{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import math\n", "import torch\n", "import qpytorch\n", "import matplotlib.pyplot as plt\n", "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Different objective functions for Approximate QEPs\n", "\n", "\n", "(*N.B.* this tutorial assumes that you are familiar with **inducing point methods**.\n", "For an introduction to these methods, please see [Quinonero-Candela and Rasmussen, 2005](http://www.jmlr.org/papers/volume6/quinonero-candela05a/quinonero-candela05a.pdf).)\n", "\n", "## Overview\n", "\n", "Approximate Q-Exponential processes learn an approximate posterior distribution\n", "\n", "$$ p(\\mathbf f(X) \\mid y) \\approx q(\\mathbf f) = \\int( p(\\mathbf f \\mid \\mathbf u) \\: q(\\mathbf u) d \\mathbf u$$\n", "\n", "where $\\mathbf u$ are the function values at some **inducing points** $\\mathbf u$.\n", "Typically, $q(\\mathbf u)$ is chosen to be a multivariate q-exponential -- i.e. $q(\\mathbf u) = \\mathcal Q (\\mathbf m, \\mathbf S)$.\n", "\n", "We choose the approximate posterior $\\int( p(\\mathbf f \\mid \\mathbf u) \\: q(\\mathbf u) d \\mathbf u$ by optimizing the parameters of $q(\\mathbf u)$ (i.e. $\\mathbf m$ and $\\mathbf S$. There are several objectives that we can use for optimization. We'll test out the following two:\n", "\n", "1. The Variational ELBO (see [Hensman et al., 2015](http://proceedings.mlr.press/v38/hensman15.pdf))\n", "2. The Predictive Log Likelihood (see [Jankowiak et al., 2020](http://arxiv.org/))." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Experimental setup\n", "\n", "We're going to train an approximate QEP on a 1D regression dataset with heteroskedastic noise." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[Text(0.5, 0, 'x'), Text(0, 0.5, 'y')]" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAckAAAEmCAYAAAAJLWsmAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAIjdJREFUeJzt3WtwVOUdx/H/nsUsCknUieHSDbGYqPVWrAgh1ooM01RbKy+KWGsaO5akNXZG6VixttLRSqjayIxDlZiIllowsaKOMlKEIF7iWBE6tColQupuMaDTmkS0CWSfvqC7bjZ7ds/Z7Lntfj8z+yLL2eThEPa3z+3/+JRSSgAAwCia0w0AAMCtCEkAAHQQkgAA6CAkAQDQQUgCAKCDkAQAQAchCQCADkISAAAd45xugJ0ikYgcOHBACgsLxefzOd0cAIADlFIyMDAgU6dOFU1L3VfMq5A8cOCAlJWVOd0MAIALhEIhCQaDKa/Jq5AsLCwUkWM3pqioyOHWAACc0N/fL2VlZbFMSCWvQjI6xFpUVERIAkCeMzLtxsIdAAB0EJIAAOggJAEA0EFIAgCgg5AEANgqHA5LZ2enhMNhp5uSFiEJALBNW1ublJeXy7x586S8vFza2tqcblJKPqWUcroRdunv75fi4mLp6+tjCwgA2CwcDkt5eblEIpHYc36/X3p6etJu6s8mM1lATxIAYIu9e/eOCEgRkeHhYenu7naoRekRkgAAW1RWVo6qler3+6WiosKhFqVHSAIAbBEMBqWlpUX8fr+IHAvI1atX2zrUahZzkgAAW4XDYenu7paKigpHAtJMFuRV7VYAgPOCwaCre4/xGG4FAEAHIQkAgA5CEgAAHYQkAAA6CEkAAHQQkgAA6CAkAQDQQUgCAKCDkAQAQAchCQDICi8dpmwUIQkAGDOvHaZsFAXOAQBj4pbDlI3i0GUAgG28eJiyUYQkAGBMvHiYslGEJABgTLx4mLJRzEkCALLC6cOUjeLQZQCA7bx0mLJRDLcCAKCDkAQAQAchCQCADkISAAAdhCQAADoISQAAdHgmJJuamuTCCy+UwsJCKS0tlQULFsiePXucbhYAIId5JiRfeuklaWxslNdff102b94sR44cka9//ety+PBhp5sGAMhRnq248+GHH0ppaam89NJL8rWvfc3Qa6i4AwDIi1NA+vr6RETk5JNPdrglAIBc5cmydJFIRG666Sa56KKL5JxzztG9bnBwUAYHB2Nf9/f329E8AECO8GRPsrGxUf72t7/J+vXrU17X1NQkxcXFsUdZWZlNLQQA5ALPzUneeOON8swzz8j27dvli1/8Ysprk/Uky8rKmJMEgDyWk6eAKKXkJz/5iWzYsEG2bduWNiBFRAKBgAQCARtaBwDIRZ4JycbGRvnjH/8ozzzzjBQWFkpvb6+IiBQXF8vxxx/vcOsAALnIM8OtPp8v6fNr1qyR6667ztD3YAsIACBnh1sBALCTJ1e3AgBgB0ISAAAdhCQAADoISQAAdBCSAADoICQBANBBSAIAoIOQBABAByEJAIAOQhIAAB2EJAAAOghJAAB0EJIAAOggJAEA0EFIAgCgg5AEAEAHIQkAgA5CEgAAHYQkAAA6CEkAAHQQkgAA6CAkAQDQQUgCAKCDkAQAQAchCQCADkISAAAdhCQAADoISQAAdBCSAADoICQBIM+Fw2Hp7OyUcDjs6Z9hBUISAPJYW1ublJeXy7x586S8vFza2to8+TOs4lNKKacbYZf+/n4pLi6Wvr4+KSoqcro5AGCZcDgse/fulcrKSgkGg7rXlJeXSyQSiT3n9/ulp6dH9zWZtMPqn2GWmSygJwkAOcZoz23v3r0jwktEZHh4WLq7u7PWFjt+hpUISQDIIeFwWOrr62PBFIlEpKGhIelcYGVlpWjayBjw+/1SUVGRtfbY8TOsREgCQA4x03MLBoPS0tIifr9fRI6F1+rVq7M6DGrHz7ASc5IAkEMymQMMh8PS3d0tFRUVloWXHT/DKDNZMM6mNgEAbBDtuTU0NMjw8LChnlswGLQ8uOz4GVagJwkAOchNPTe3oScJAHnOqz03t2HhDgAAOghJAAB0EJIAAOggJAEA0EFIAgCgg5AEAECHp0Jy+/btcsUVV8jUqVPF5/PJ008/7XSTAAA5zFMhefjwYfnyl78sq1atcqwNiQeHpvsaAOBdniomcNlll8lll13m2M9va2uLVdfXNE1qa2tl7dq1ul+vWLFCZs6cmfI8NwCAe3m2LJ3P55MNGzbIggULdK8ZHByUwcHB2Nf9/f1SVlaWUVm6ZEWDjdI0TVpaWqSmpibtIagAgM8ZOTzaLA5d/r+mpiYpLi6OPcrKyjL+XsmOnzEqEonI4sWLRxyCeu+99zIsCwApGD082kr0JA0aS08yFXqZANzMip6c0Z9r9sgvo+hJ/l8gEJCioqIRj0wlOzi0rq5O92uj6GUCyJZsLxx0sidn5vBoSymPEhG1YcMGU6/p6+tTIqL6+voy/rmhUEh1dnaqUCiU8ut7771X+f1+JSJK0zTl8/mUiJh6aJqmWltbM24rgPzR2tqqNE3L2ntHKBSKfb/ow+/3x97rrGblzzeTBZ4abv3kk09inyLOP/98aW5ulksvvVROPvlkmTZtWtrX232eZPx5bps2bYodgmqG3++Xrq4u+eSTTxiKBZCUFUOTnZ2dMm/evKTPz507N9OmmtLW1jbq8Ojrr79+zN/XTBZ4KiS3bdsml1566ajn6+rq5NFHH037eqcPXY6G5ptvvilLly6V4eFh0TRNlFKS6p9B07TYtpKWlpas/JIAyB1WBJqVc4Jm25Htw6NzNiTHyumQjJdpL1PTNFm3bp1UV1fTqwQgItYFmlU9OacRkjrcFJKJ9HqZeqtp6VUCiGdVoFnRk3MaIanDzSEZL/pLOWHCBKmqqtINSuYrAcTLxUCzAiGpwyshGS/+02EyzFcCgDmEpA4vhqTIsU+HXV1dcvXVV6csZkDPEgDSo5hAjgkGg7Jw4cIRxQw0bfQ/3fDwsFRVVTlawgkAxspNpykRkh5y/fXXS09Pj3R2dsrrr7+eNCijPc1IJCL19fXS3t7uil80ADDCDfVa4zHc6mHx85WshAXgdXbtzWS4NU8Y6VmKHOtVNjQ00KMEYJtMhkxdU681jumQrKurk+3bt1vRFmQgGAzK3Llz5cILLxwxZ5loeHhYurq6XDPODyB3ZTpkWllZOerDvt/vl4qKCiuaaYjpkOzr65P58+dLZWWlLF++XP71r39Z0S5kINqzbG9vH/WL5vP55Oqrr3bNOD8AY9y0iMWIcDgs9fX1I9ZHGB3JSnba0urVq51dqZ9JBfVDhw6p3/72t+q8885T48aNU9/4xjdUR0eHGhoayuTb2SYbp4B4RWtra8pTSDRNU0888YRtFf0BGBMKhdTWrVtVKBTK+skedti6dWvSU406OzsNf4/E05WyzdZTQN566y1Zs2aNtLa2ysSJE+Xaa6+VG264QSorK8fybS2Rawt30olW3zh06JAsWrQo6TUs6gHco62tLdYL8/l8IiIjDj9wosC4WW4pjJ6KbQt3PvjgA9m8ebNs3rxZ/H6/XH755bJ7924566yz5P777x/Lt0YWROcrq6urWdQDuFziMKVKcjqQ04tYjHDlkOkYmA7JI0eOyJ/+9Cf51re+JeXl5dLR0SE33XSTHDhwQB577DF58cUXpb29Xe68804r2osMJP7SJmJRD+C8ZCs7Ezm9iMWo+JX3PT09nh6pMj3cWlJSIpFIRL773e/K4sWLZcaMGaOu+fjjj+X888+X/fv3Z6udWZFvw62J9Mrb+Xw+8fl81IAFHJRsmNLn84mmabYfVRUOh2Xv3r05W97SVBaYnfD8/e9/rz777DOzL3OFfFq4k0q6RT1+v58FPYAD4v9v+v1+1dramnIRS/win3SMXuvFxUJmmcmCjFa3ehUh+bnof7wnnngi6Uq05uZmghKwSKrAMrqy00yYGb02FArFrrPzQ7OZsM8GQlIHITlasv8U0UeufooEnJSNnpqZMDNzbTa2b5jlRM/VTBZQli7PpVrUw8pXILvGstE+npnybWautbviTbbuh5UIScRWojU3N4/6M1a+AtmTrdqkZsLMzLV2b99wY63WRIQkROTzMyspZwdYJ1s9NTNhZjb47Ny+4cZaraNYPvjrIsxJpsfKV8BayVawZspM+TarS71lKpv3wyhby9J5Sb7vkzQqXTm7zs5OmTt3rv0NA3JE9P9YRUVFTu5DNMvu+2EmC8ZZ3hp4TjAYlGAwKOFweNRhzpqmyaFDhyQcDvOfG8hQ9P8YjnHz/WBOEroS5zJ8Pp8opWTRokXMTyJvee3oqmyL//vnw70gJJFS/BmV0ZAUcedSbcBqmR4mnCvi//7Tpk2TadOm5fy9YE4ShnR2dsq8efNGPd/e3i4lJSU5W+MRiDJyBFQu1zxN9veP57bjsFKx7ags5I9kS7XZHoJ8km5PX673MtOdUuK2/Y3ZQkjCkMT5yWhgurlSBpBNqfb0eaFyzFgl+/vHc93+xiwhJGFY/CbjdevWJT0Qluo8yFWpNuV7oXLMWCVbyBcNTa8frJwKc5LIiN7Zd5xLiVyXbE+fkfnKXBH/9xcRT+73NJMFhCQy1tbWJg0NDTI8PCyapok6dqpM7M9z9U0CSCb+/4OdByRnIpcXGBlBSOogJLOP6jzA57xQSaetrS02f5qvIz6EpA5C0jrJhps0TZN169ZJdXW1a98wgHyST8PCqbAFBLajOg+Qmhuq0+TDAqNsIySRNVTnAZJzyx5KTxxNlYbdHzYISWRVMBiUkpKSpJ9W2R6CfOSmPZR2H6qcbU582GBOElnH9hDgc3olHZ1c1OaFBUaJsjmfypwkHEV1HuBzbhziDAaDMnfuXM8EpIhz86mEJCxhpDpPR0cHQYmc5/UhTrdw6sMGw62wXKrTAxh6Rb7w4hCn22SrYAP7JHUQks6J/+VOlI/7tABkJhsfNpiThOtEh1+bm5tH/Rn7tAAYZfd8KiEJ2wSDQVm4cGHSeYUJEyawPQSA63guJFetWiWnnnqqjB8/XmbPni1vvPGG002CCckWMVx77bVSVVXl+EZrAEjkqTnJJ554Qr7//e/LQw89JLNnz5aVK1dKR0eH7NmzR0pLS9O+njlJ94jOK0yYMEGqqqqo+QrANjk7J9nc3CyLFy+WH/zgB3LWWWfJQw89JCeccII88sgjTjcNJkXnFT755JNRq14jkQg1XwG4gmdCcmhoSHbs2CHz58+PPadpmsyfP1+6urqSvmZwcFD6+/tHPOAuyfY+RVF0AIDTPBOSH330kQwPD8ukSZNGPD9p0iTp7e1N+pqmpiYpLi6OPcrKyuxoKkxInKNMxMpXAE7yTEhm4rbbbpO+vr7YIxQKOd0kJBF/eojbyncByG+eCcmSkhLx+/1y8ODBEc8fPHhQJk+enPQ1gUBAioqKRjzgTtHtIcnKd4kI20MAOMIzIVlQUCAXXHCBbNmyJfZcJBKRLVu2yJw5cxxsGbIpvuZrT0+PiIgrzuEDkJ88twWkrq5OVq9eLbNmzZKVK1dKe3u7vPvuu6PmKpNhC4i3ZPNoHACIMpMF42xqU1YsWrRIPvzwQ7njjjukt7dXZsyYIS+88IKhgIT36B2N09HRIQsXLiQoAVjOUz3JsaIn6S2cHgLACjlbTAD5JdX2EPZQwu3C4TALznIAIQlXS3d6SFdXF29EcJ22tjYWnOUIhlvhCcmGXn0+n/h8PolEIgy/wjLhcFj27t0rlZWVhubBWXDmfgy3IuckDr1Giw5E34gYfoUV0vUIkw2p6i04o3KUNxGS8Iz4PZTr1q2TxEEQ3oiQTeFwWOrr63U/iOkFaLJ6xFSO8i5CEp4SPT2kurqaNyJYKlWPMFWAJjszdfXq1Qy1ehQhCU/SeyMSoYQdsiNVjzDdkGpi5ah0c+WshHUvQhKeRQk7WClVj9DIkGp01CNdD5KVsO7G6lbkBFYUIpHZVal6rwuHw9Ld3S0VFRUjvk9bW5s0NDTI8PBwLEDNrq7m99YZrG5F3klVwo4hrPyTae8s2ev0eoRmh1STYSWs+9GTRE4wUsKupqYmo54FvCXT3pkTvTp6ks6gJ4m8k66E3eLFi5n3yROZ9s6c6NWxEtb96Ekip4TDYeno6JAlS5akvI5P67nLSz3J+J+dbN4T1qAnibwVDAZl4cKFo1YeJmLeJ3dl2jtzsldndCUs7EdPEjkpfuWhpmmilBpRoYeeZO7LtHdGry73mckCQhI5K/7NbtOmTSOW6zc1NcnMmTNZxAPkIUJSByGZ36Kh+eabb8qtt97K6SE5yOjeyEz3UCI3MCcJJBEMBqWioiIWkCKcHpJLjO6NpMINzCAkkVfYvJ2b0p3YYfY6IIqQRF7hGKPcZPTDDx+SYBYhibzC6SG5yeiHHz4kwSxCEnmH00Nyj9E9jlS4gVmsbkVeo3ZmbjG6x5G9kPnNTBaMs6lNgCulOj1k4cKFvIF6TDAY1P03S9z2wb8tjGC4FXkt2RyViMiSJUtiQ6+cGu99bPtAphhuRd6LL2GXyOfzic/no/CAi5gtBMCQOhJRTAAwIbqQp7m5edSfKaXYU+cimfQI2faBsSAkAeH0EC9IVgigvr5e2tvbU35wYdsHxoKQBP4vcXuApmni8/lGXKNpmhw6dCj2psx8pX2S9QgjkYgsWrQoZa+SbR8YC+YkgQR6p4dEA1MpJZqmSW1traxdu5b5Spskm1uMl26ekW0fiOIUEB2EJDIRDoelq6tLrr76at03aBEWg9gh1SIrkWNVk+bOnWtvo+A5LNwBsigYDEpJSUnKgBRhvtIO0UVW7e3tzDPCFoQkYIDefsp4fr9fJkyY4Ik5yrHMpTo9DxtdZJWv84xO3/+8o/JIX1+fEhHV19fndFPgQa2trcrv9ysRUX6/X9XV1Y36WtM0JSJK0zTV2to64vWhUEht3bpVhUIhh/4Gx7S2tqZsp1WvtUIoFFKdnZ2O31O7uO3+e5WZLGBOEjAhcfFH9OsJEyZIVVXViCFZTdNk3bp1Ul1dLZs2bYptX3Bykc9YNtZbuSk/vkCAiJgqFpAvKIqQPdRuBSySWPMz+nVnZ6fu9oT4VbHR5xsaGqSmpsb2N7dUG+vTtWUsr02lra0t9gEicQVxS0uL1NTUEJpi3f1HasxJAlmQas5SKSWJAzbRIup2zyuNZWO9FZvyEwsExN+rSCQiixcvpubq/1EUwRmEJJAFiRvWjYgvom6XsWyst2JTfrLeUTxFWcAYiiI4gzlJIIv09lT6fD7RNC3p/j4n5pXGsrHezJmN6YZJ0xUISCbf90JSFGHs2CcJOERve8LDDz+sW0Tdif2VwWBQ5s6dm9GbrJHXGi1Entg7in6YEEleFpDhxbH928E8epKARZJ94s+HFYqZ/B3j75WIJC0LGB1etGJVsNnjt+BtrG4FXCBxJWz0uZaWllFv/LnyxhwOh6W9vd30Ksxkq4ZFjlXYqampsXR4MX51LTV4kYieJOCAVPNKXu3VxIdNIrf2lvOhZ4/RmJMEXE5vXimTQ4WzYaylzhK3csRzc2+ZA5mRDiEJuESyQ4Xt2PKQjWDW28px//33S09Pj2uHL9l7iHQ8E5J33323VFdXywknnCAnnnii080Bss6JXo2RYDbSy9QLm+985zuu7EFGsfcQ6XgmJIeGhmThwoXy4x//2OmmAJYw2qsxOjRq5Lp0wZzpVg4vhU30+K3Ozk5X93rhECsqrFtpzZo1qri4OKPXcgoI3C7xpJHEUx6MngJh9LpQKBS7Lvrw+/0qFAql/DM9+XYqB7zJTBZ4pieZicHBQenv7x/xANwo2uurqanR7dUYnbM0M7eZqgeYyfAvG92Ra3I6JJuamqS4uDj2KCsrc7pJwCiJQ5qbNm1KGjRGQ8tsuOkNN7KoBXA4JJcuXSo+ny/l49133834+992223S19cXe4RCoSy2Hhg7M70+o6GVSbgl6wF6eZ4RyBZHK+789Kc/leuuuy7lNdOnT8/4+wcCAQkEAhm/HrCamTMCjVTriRYi+M1vfiNLly4dcZ3IseLgZooU2FHxBnAzR0PylFNOkVNOOcXJJgCOivb6Eiu+6PX6UoVWYnm1FStWyIUXXhirgRqtLGO29Fqy8npAvvBMWbr3339f/v3vf8uzzz4r9957r7z88ssiIlJRUSETJ0409D0oSwc3amtrG3MR71Tl1UTEVOk1r5bFA4zKyQLnd9xxhzz22GOxr88//3wR4Ww5eF+6IU0joZVq2FbFHVyc+GfJyuJR7Bv4nGd6ktlATxJeky60ogE6ceJEqaqqGlNPkmLfyBcUOAdyQLqVr/FbR6qqqqS2tjbpSlSjq1Qz2ReZaWH0sRZUB+zimeFWIN+kC63EAP3DH/4gXV1dcvjw4VHDtkZWqZpdRGR2aDba692xY4fceuutDOnCG6wt/uMulKWDl6QqC7d169YRz0cfnZ2dY/qZ6criGWmb3vdNvN7I6wArUJYOyAGphkmtqoZjtNi3maHZVGdNpnod4AYMtwIupjdMaqSwQKaM7Is0MzSrd9ZkutcBbkBIAi6nF1pOVsMxE9LJAjWKUndwO7aAAMhYOBw2FNKJBROamppi1YAISNjNTBYQkgBsYTRQAavlZMUdAN5GDVh4EatbAQDQQUgCAKCDkAQAQAchCQCADkISAAAdebW6Nbrbpb+/3+GWAACcEs0AIzsg8yokBwYGRESkrKzM4ZYAAJw2MDAgxcXFKa/Jq2ICkUhEDhw4IIWFheLz+TL6Hv39/VJWViahUIiCBAm4N8lxX/Rxb5LjvujLxr1RSsnAwIBMnTp11EEBifKqJ6lpWtY2MxcVFfHLq4N7kxz3RR/3Jjnui76x3pt0PcgoFu4AAKCDkAQAQAchaVIgEJBly5ZJIBBwuimuw71Jjvuij3uTHPdFn933Jq8W7gAAYAY9SQAAdBCSAADoICQBANBBSAIAoIOQTGLVqlVy6qmnyvjx42X27NnyxhtvpLy+o6NDzjzzTBk/fryce+65snHjRptaaj8z9+bhhx+Wiy++WE466SQ56aSTZP78+WnvpVeZ/Z2JWr9+vfh8PlmwYIG1DXSQ2Xvz8ccfS2Njo0yZMkUCgYCcfvrpOfl/yux9WblypZxxxhly/PHHS1lZmdx8883y3//+16bW2mP79u1yxRVXyNSpU8Xn88nTTz+d9jXbtm2Tr3zlKxIIBKSiokIeffTR7DZKYYT169ergoIC9cgjj6i///3vavHixerEE09UBw8eTHr9q6++qvx+v7rnnnvU22+/rX7xi1+o4447Tu3evdvmllvP7L255ppr1KpVq9TOnTvVO++8o6677jpVXFyswuGwzS23ltn7ErV//371hS98QV188cXqyiuvtKexNjN7bwYHB9XMmTPV5Zdfrl555RW1f/9+tW3bNrVr1y6bW24ts/fl8ccfV4FAQD3++ONq//79atOmTWrKlCnq5ptvtrnl1tq4caO6/fbb1VNPPaVERG3YsCHl9fv27VMnnHCCWrJkiXr77bfVAw88oPx+v3rhhRey1iZCMsGsWbNUY2Nj7Ovh4WE1depU1dTUlPT6q666Sn3zm98c8dzs2bNVQ0ODpe10gtl7k+jo0aOqsLBQPfbYY1Y10RGZ3JejR4+q6upq1draqurq6nI2JM3emwcffFBNnz5dDQ0N2dVER5i9L42NjWrevHkjnluyZIm66KKLLG2nk4yE5M9+9jN19tlnj3hu0aJFqqamJmvtYLg1ztDQkOzYsUPmz58fe07TNJk/f750dXUlfU1XV9eI60VEampqdK/3qkzuTaJPP/1Ujhw5IieffLJVzbRdpvflzjvvlNLSUrn++uvtaKYjMrk3zz77rMyZM0caGxtl0qRJcs4558jy5ctleHjYrmZbLpP7Ul1dLTt27IgNye7bt082btwol19+uS1tdis73n/zqsB5Oh999JEMDw/LpEmTRjw/adIkeffdd5O+pre3N+n1vb29lrXTCZncm0S33nqrTJ06ddQvtZdlcl9eeeUVaWtrk127dtnQQudkcm/27dsnW7dule9973uyceNG6e7ulhtuuEGOHDkiy5Yts6PZlsvkvlxzzTXy0UcfyVe/+lVRSsnRo0flRz/6kfz85z+3o8mupff+29/fL5999pkcf/zxY/4Z9CRhixUrVsj69etlw4YNMn78eKeb45iBgQGpra2Vhx9+WEpKSpxujutEIhEpLS2VlpYWueCCC2TRokVy++23y0MPPeR00xy1bds2Wb58ufzud7+Tt956S5566il5/vnn5a677nK6aTmPnmSckpIS8fv9cvDgwRHPHzx4UCZPnpz0NZMnTzZ1vVdlcm+i7rvvPlmxYoW8+OKLct5551nZTNuZvS/vvfee9PT0yBVXXBF7LhKJiIjIuHHjZM+ePXLaaadZ22ibZPI7M2XKFDnuuOPE7/fHnvvSl74kvb29MjQ0JAUFBZa22Q6Z3Jdf/vKXUltbKz/84Q9FROTcc8+Vw4cPS319vdx+++1pz0TMVXrvv0VFRVnpRYrQkxyhoKBALrjgAtmyZUvsuUgkIlu2bJE5c+Ykfc2cOXNGXC8isnnzZt3rvSqTeyMics8998hdd90lL7zwgsycOdOOptrK7H0588wzZffu3bJr167Y49vf/rZceumlsmvXLikrK7Oz+ZbK5Hfmoosuku7u7tgHBxGRf/zjHzJlypScCEiRzO7Lp59+OioIox8kVB6X37bl/TdrS4ByxPr161UgEFCPPvqoevvtt1V9fb068cQTVW9vr1JKqdraWrV06dLY9a+++qoaN26cuu+++9Q777yjli1bltNbQMzcmxUrVqiCggL15JNPqg8++CD2GBgYcOqvYAmz9yVRLq9uNXtv3n//fVVYWKhuvPFGtWfPHvXcc8+p0tJS9etf/9qpv4IlzN6XZcuWqcLCQrVu3Tq1b98+9ec//1mddtpp6qqrrnLqr2CJgYEBtXPnTrVz504lIqq5uVnt3LlT/fOf/1RKKbV06VJVW1sbuz66BeSWW25R77zzjlq1ahVbQOzwwAMPqGnTpqmCggI1a9Ys9frrr8f+7JJLLlF1dXUjrm9vb1enn366KigoUGeffbZ6/vnnbW6xfczcm/LyciUiox7Lli2zv+EWM/s7Ey+XQ1Ip8/fmtddeU7Nnz1aBQEBNnz5d3X333ero0aM2t9p6Zu7LkSNH1K9+9St12mmnqfHjx6uysjJ1ww03qP/85z/2N9xCnZ2dSd8zoveirq5OXXLJJaNeM2PGDFVQUKCmT5+u1qxZk9U2cVQWAAA6mJMEAEAHIQkAgA5CEgAAHYQkAAA6CEkAAHQQkgAA6CAkAQDQQUgCAKCDkAQAQAchCQCADkISyAMffvihTJ48WZYvXx577rXXXpOCgoJRpygA+By1W4E8sXHjRlmwYIG89tprcsYZZ8iMGTPkyiuvlObmZqebBrgWIQnkkcbGRnnxxRdl5syZsnv3bvnLX/4igUDA6WYBrkVIAnnks88+k3POOUdCoZDs2LFDzj33XKebBLgac5JAHnnvvffkwIEDEolEpKenx+nmAK5HTxLIE0NDQzJr1iyZMWOGnHHGGbJy5UrZvXu3lJaWOt00wLUISSBP3HLLLfLkk0/KX//6V5k4caJccsklUlxcLM8995zTTQNci+FWIA9s27ZNVq5cKWvXrpWioiLRNE3Wrl0rL7/8sjz44INONw9wLXqSAADooCcJAIAOQhIAAB2EJAAAOghJAAB0EJIAAOggJAEA0EFIAgCgg5AEAEAHIQkAgA5CEgAAHYQkAAA6CEkAAHT8D3fRfLS1HXoxAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Define some training data\n", "train_x = torch.linspace(0, 1, 100)\n", "train_y = torch.cos(train_x * 2 * math.pi) + torch.randn(100).mul(train_x.pow(3) * 1.)\n", "\n", "fig, ax = plt.subplots(1, 1, figsize=(5, 3))\n", "ax.scatter(train_x, train_y, c='k', marker='.', label=\"Data\")\n", "ax.set(xlabel=\"x\", ylabel=\"y\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here's a simple approximate QEP model, and a script to optimize/test the model with different objective functions:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "POWER = 1.0\n", "class ApproximateQEPModel(qpytorch.models.ApproximateQEP):\n", " def __init__(self, inducing_points):\n", " self.power = torch.tensor(POWER)\n", " variational_distribution = qpytorch.variational.CholeskyVariationalDistribution(inducing_points.size(-1), power=self.power)\n", " variational_strategy = qpytorch.variational.VariationalStrategy(\n", " self, inducing_points, variational_distribution, learn_inducing_locations=True\n", " )\n", " super().__init__(variational_strategy)\n", " self.mean_module = qpytorch.means.ConstantMean()\n", " self.covar_module = qpytorch.kernels.ScaleKernel(qpytorch.kernels.RBFKernel())\n", "\n", " def forward(self, x):\n", " mean_x = self.mean_module(x)\n", " covar_x = self.covar_module(x)\n", " return qpytorch.distributions.MultivariateQExponential(mean_x, covar_x, power=self.power)" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "# this is for running the notebook in our testing framework\n", "import os\n", "smoke_test = ('CI' in os.environ)\n", "training_iterations = 2 if smoke_test else 200\n", "\n", "\n", "# Our testing script takes in a QPyTorch MLL (objective function) class\n", "# and then trains/tests an approximate QEP with it on the supplied dataset\n", "\n", "def train_and_test_approximate_qep(objective_function_cls):\n", " model = ApproximateQEPModel(torch.linspace(0, 1, 100))\n", " likelihood = qpytorch.likelihoods.QExponentialLikelihood(power=model.power)\n", " objective_function = objective_function_cls(likelihood, model, num_data=train_y.numel())\n", " optimizer = torch.optim.Adam(list(model.parameters()) + list(likelihood.parameters()), lr=0.1)\n", "\n", " # Train\n", " model.train()\n", " likelihood.train()\n", " for _ in range(training_iterations):\n", " output = model(train_x)\n", " loss = -objective_function(output, train_y)\n", " loss.backward()\n", " optimizer.step()\n", " optimizer.zero_grad()\n", "\n", " # Test\n", " model.eval()\n", " likelihood.eval()\n", " with torch.no_grad():\n", " f_dist = model(train_x)\n", " mean = f_dist.mean\n", " f_lower, f_upper = f_dist.confidence_region()\n", " y_dist = likelihood(f_dist)\n", " y_lower, y_upper = y_dist.confidence_region()\n", "\n", " # Plot model\n", " fig, ax = plt.subplots(1, 1, figsize=(5, 3))\n", " line, = ax.plot(train_x, mean, \"blue\")\n", " ax.fill_between(train_x, f_lower, f_upper, color=line.get_color(), alpha=0.3, label=\"q(f)\")\n", " ax.fill_between(train_x, y_lower, y_upper, color=line.get_color(), alpha=0.1, label=\"p(y)\")\n", " ax.scatter(train_x, train_y, c='k', marker='.', label=\"Data\")\n", " ax.legend(loc=\"best\")\n", " ax.set(xlabel=\"x\", ylabel=\"y\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Objective Funtion 1) The Variational ELBO\n", "\n", "The **variational evidence lower bound** - or ELBO - is the most common objective function. It can be derived as an lower bound on the likelihood $p(\\mathbf y \\! \\mid \\! \\mathbf X)$:\n", "\n", "$$ \\mathcal{L}_\\text{ELBO} = \\sum_{i=1}^N \\mathbb{E}_{q( \\mathbf u)} \\left[ \\mathbb{E}_{f( \\mathbf f \\mid \\mathbf u)} \\left[ \\log p( y_i \\! \\mid \\! f_i) \\right] \\right] - \\beta \\: \\text{KL} \\left[ q( \\mathbf u) \\Vert p( \\mathbf u) \\right]$$\n", "\n", "where $N$ is the number of datapoints and $p(\\mathbf u)$ is the prior distribution for the inducing function\n", "values. For more information on this derivation, see [Hensman et al., 2015](http://proceedings.mlr.press/v38/hensman15.pdf).\n", "\n", "### How to use the variational ELBO in QPyTorch\n", "\n", "In QPyTorch, this objective function is available as [qpytorch.mlls.VariationalELBO](https://qepytorch.readthedocs.io/en/stable/marginal_log_likelihoods.html#variationalelbo)." ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAckAAAEmCAYAAAAJLWsmAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAY7RJREFUeJztnXd4VGX2x793bjKTNuk9kwIkoBSpgkSRIm7EvjZsCBokKlZ2VdTfyrruCroui7qoRCMgIioqoICoYLAgiDSlCQmQhEmF9J4p9/fHyc2kTZLpmeR8nuc+hMmdmXdu7rzf95z3FEGSJAkMwzAMw3RA4eoBMAzDMExvhUWSYRiGYczAIskwDMMwZmCRZBiGYRgzsEgyDMMwjBlYJBmGYRjGDCySDMMwDGMGFkmGYRiGMYOHqwfgTIxGIwoKCqBWqyEIgquHwzAMw7gASZJQXV2N6OhoKBRd24r9SiQLCgoQGxvr6mEwDMMwvYCzZ89Co9F0eU6/Ekm1Wg2ALoy/v7+LR8MwDMO4gqqqKsTGxrZoQlf0K5GUXaz+/v4skgzDMP2cnmy7ceAOwzAMw5iBRZJhGIZhzMAiyTAMwzBmYJFkGIZhGDOwSDIMwzBORavVIjMzE1qt1tVD6RYWSYZhGMZpZGRkID4+HtOmTUN8fDwyMjJcPaQuESRJklw9CGdRVVWFgIAAVFZWcgoIwzCMk9FqtYiPj4fRaGx5TBRF5OTkdJvUb08s0QK2JBmGYRinkJWV1UYgAcBgMCA7O9tFI+oeFkmGYRjGKSQlJXWolSqKIhITE100ou5hkWQYhmGcgkajQXp6OkRRBEACuWLFCqe6Wi2F9yQZhmEYp6LVapGdnY3ExESXCKQlWtCvarcyDMMwrkej0fRq67E17G5lGIZhGDOwSDIMwzCMGVgkGYZhGMYMLJIMwzAMYwYWSYZhGIYxA4skwzAMw5iBRZJhGIZhzMAiyTAMwzBmYJFkGIZh7II79YnsKSySDMMwjM24W5/InsK1WxmGYRib6C19InsK95NkGIZhnIY79onsKSySDMMwjE24Y5/InsIiyTAMw9iEO/aJ7Cm8J8kwDMPYBVf3iewp3E+SYRiGcTru1Ceyp7C7lWEYhmHMwCLJMAzDMGZgkWQYhmEYM7BIMgzDMIwZ3EYkFy9ejIsvvhhqtRrh4eG48cYbceLECVcPi2EYhunDuI1Ifv/995g/fz727NmDb7/9FjqdDn/6059QW1vr6qExDMMwfRS3zZM8d+4cwsPD8f333+Pyyy/v0XM4T5JhGIbpF7VbKysrAQDBwcEuHgnDMAzTV3HLYgJGoxGPP/44Lr30UgwfPtzseY2NjWhsbGz5f1VVlTOGxzAMw/QR3NKSnD9/Po4cOYKPPvqoy/MWL16MgICAliM2NtZJI2QYhmH6Am63J/nwww9j06ZN+OGHHzBgwIAuz+3MkoyNjeU9SYZhmH5Mn6zdKkkSHnnkEWzYsAE7d+7sViABQKVSQaVSOWF0DMMwTF/EbURy/vz5+PDDD7Fp0yao1WoUFRUBAAICAuDt7e3i0TEMwzB9EbdxtwqC0OnjK1euxJw5c3r0GpwCwjAMw/RZdyvDMAzDOBO3jG5lGIZhGGfAIskwDMMwZmCRZBiGYRgzsEgyDMMwjBlYJBmGYRjGDCySDMMwDGMGFkmGYRiGMQOLJMMwDMOYgUWSYRiGYczAIskwDMMwZmCRZBiGYRgzsEgyDMMwjBlYJBmGYRjGDCySDMMwDGMGFkmGYRiGMQOLJMMwDMOYgUWSYRiGYczAIskwDMMwZmCRZBiGYRgzsEgyDMMwjBlYJK1AkgCDwdWjYBiGYRyNh6sH4I6UlgIVFYCHB6BU0uHpSYeHB/0rCK4eJcMwDGMrLJJWoNcDDQ2AlxfQ2Ej/l5GF0seHfq9UAioVIIquGy/DMExvxmgk75wk0WE00r+A6V/AZHzIRokzYJG0ElkIWyNJJJh6PVBeTn90hYKE0tsb8PUlwfTyYkuTYZj+g8Fgmhv1evp/UxMder1JGGVxbC+S7efL4GAgKso5Y2eRtCOCYFrheHvTY0Yj3QjV1SScokgi6e9P53h5kZAyDMP0BQwGQKcziWBdHf1fp2sbyyGKNPeJomnuVCjoZ3lO7MyYqK5ua106GhZJB6NQkBB6edH/DQZy0RYV0Q3g5QUEBJjcs2xhMgzjThiNNKc1NQG1tUB9Pf1sNNLvW29Bebih4rjhkN0bUaSbxceHbqKGBhJMhYIeCwwkt6w73kwMw7gnWq0WWVlZSEpKgkaj6fZ8OS6jvh6oqiJRNBhMgYt+fh3jMAoKtDhzJgsDBiQhOrr79+gtsKPPhcjCGBREwtjQAJw9C+TkAOfO0Q3IMAxjDVqtFpmZmdBqtV2el5GRgfj4eEybNg3x8fHIyMjo9DyjEaipAYqLaY7KzaV5CiBRDA42bSO1F8h16zIwYUI8brttGiZMiMe6dZ2/R29EkCRnenddS1VVFQICAlBZWQl/f3+rX6eoiFJAbHgJs0iSaYXm4UHvIbtj2RXLMExPyMjIwLx582A0GqFQKJCeno7U1NQO52m1WsTHx8Mo+0YBiKKInJycFouyoYHEsbKS3KqSZNpC6smcVFCgxYQJHd9jz54cqyzK6mpArQaioy1+aguWaAFbkr0MQaCVWHAw/VtRQSs2rZZu1P6zpGEYxhq0Wm2LQAKA0WhEWlpapxZlVlZWG/ECAIPBgJMns1FdDeTnk9VYXExzj78/eb68vXu+aD9zpvP3yMnJturzORve+erFeHrSHqXBQBvi8gpKds+yZckwTHvMCV92dnaH/cakpCQoFIoOVp6XVyLy8kxbQrbkJA4Y0Pl7JCQkWv+iToQtSTdAFGkFp1aTWObl0Qqvrs7VI2MYprchC19rRFFEYmJHUdJoNEhPT4fYvImoUIh4/vkVCAvTICCAtnpsTdqPjtbglVdM7yGKIl5+eYXbBO/wnqQVOHJPsicYDGRVCgJZlYGBphQThmGYjIwMpKWlwWAwQBRFrFixotM9Sb2e5pKjR7XIyspGYmIiBg7UOMRLVVCgRU5ONhISEm0SSGfvSbJIWoGrRVJGp6N9Sk9P2sMMDOTUEYZhCK1Wi+xsEr72blZ5oV1aSkGCcmETd9jCqaoikYyJseU1eq4FPKW6MZ6eZEnKuZbV1UBICN1A7nCzMwzjODQaTQdxlCRaWJeW0r9eXjSH9Lb5QpJIvGtr2x4VFUBJCTBoEHDzzc4ZC4tkH8DLi2rC1tZSFGxgIImlSuXqkTEM01uorwfKyiiVQxRpnugNJTGNRlrgV1XRUV4OnD9P81lDA3nMABqrpycJZUSE88bHItlHEARK6JWLq9fWAqGhtPHeG74IDMO4BoOB5oSyMpof/Pxcuy2j05HQyWJYXEwiWV9PFqSHh8n9GxjYMXCosdG542WR7GN4eJD7pL6eImBrakgs5YLrDMP0H2pqSIhqaiiVw8/P+WMwGkkUy8qAggLaGqqpIbETRUpn8/cn67C3uX0BFsk+i7c3uVurqkgww8LYqmSY/oJOR6JUWkpC5Ox9R53OZCXm5ZHVWFdHi3g/P5qP3GU7iEXSCvLzae8vONhUnqk3dvFQKMhdUV9PK7i6OrIq3eXmZBjGMuTAHHlPT612XnNinY6CagoKqEpYRQU95udHC/TIyN41P/YUtxLJH374Af/+97+xf/9+FBYWYsOGDbjxxhudPo7sbGDPHlMUqVJpaqYcGEjiqVabCgC4OofR25vGWFFBghkezhGwDNPX0OvJejx/3nnWo9FI71dQAJw+TZar0UjzS1SU8wTakbiVSNbW1mLkyJG47777cNNNN7lkDIcOFWDfvlMwGAZg4EANSku1yM/PQmBgEiRJg8pK4ORJOlepJIEKCaEbJijIVFLO2chfmro6soJDQujgvEqGcX/q6qgjR3U1WW5KpePfLz+fDIaiIopC9fenBP++IIytcaspcsaMGZgxY4bL3j8jIwNz5xoAzANQDl/fHNTW7gNwAsA7uPHGm5GcPAFVVVmIiEiCTgecPp2F4uIknD5NVSzUavLHazTk+gwJce4+oVyH8dw5urHDwzmoh2HcFaORUjpKSuhnR1qPkkRWY04OiWNFBXnJ5GYMfRW3EklLaWxsRGOreOGqqiqrX0uurA981PxIEGprgwAktJyzcSOwcWMjAC2ATwHsBvATBKEIc+akY+jQFOTmkmhmZ2vg5UUiOXAgRXY5SzDlIgSVldS/MiKCVoHsfmUY90EOjikrI7FylFAZDCarMTeXFtdBQTRvOWPOKCvToriYDI/gYOfXe+3TIrl48WK88MILdnktU2X92wD4gMRxIIBBAC4EMALAcAD+AC5pPh4HAEjSMaxcuR3AfADbIQgNuPXWJYiOHoeamiTk55NghoUBSUnkslCr7TJsswiCKagnP59u/NDQjs1SGYbpfdTWkvVYW0sLXEdsm+h0FJl64gTNEQDNUT4+9n8vc/zwQwZWrZoHSTJCEBSYMycdGk3HGrSOxG1rtwqC0G3gTmeWZGxsrFW1WztrTto5A0ECObH5GAWgtfLUAfgGwAYAX0IQKjFnTjoSE1Nw6lQWfH2TEBenQUICkJBAEWGOti51OtrLCAgg9ytHvzJM70SSTKXZJMkxAXg6HblUjx0DCguBhgYtDIYsxMQ415IrK9Pir3+NhySZ5lyFQsQjj+Rg9GgNZs+2/rW5dmszKpUKKjvN+HJLGbmyviCIGDnybvz22weQJAMEQQFAgiSdBnAawIfNzwwEMA3AlQBSAAwAcGPzoYMkfYWVKz8A8AiAegiCAtdeuwS//z4O4eFJGDZMg6Qk2sN01Ga83LeyqgpoaiL3qyuSjhmGMY/BQO7V8+fNu1cLCrQ4cyYLAwYkWdxpQ68nd+rRoxSt6uUF5OZmYM2atpbc5ZdbZskZjZSWUlVlKjOn09H7iSLNPx4e9H5qNR0eHkBxcVYbgaTXMqCsLBuA88S6T1uS7bFHFxCtVoutW7Oh0yVCrdYgJ0cLrTYbvr6JyMr6Glu3pjWLpgBAaLm5SEAlABcB+HPzMbLVK1cAWAfgLQCHmz+jAtdfn46RI1MREQEMHQrExzvW0qupoZs6IoKEk/cpGcb1NDaS9VhZaT73cd26DDz11DwYjUYoFAq88ko67rije0EzGini/cgRilFQKun7X1XVuSX373/ndLAoJYnSP3JyyPosKqJCAnLEraUq4+MDBAY2oaBgM4BsUHDkIQjCH3j00RNOtSTdSiRramqQnZ0NABg9ejSWLl2KqVOnIjg4GHFxcd0+3xGtsoxG2heoqqIb+MQJLY4fz4a3dyKamoDKymzExyciN/drrFmTBqPR0OqVLgQwC8BdAFqP/yeQWK6HQmHE00/vRklJDXx9kzBkiAYXXkiuWEeJZUMDhXiHhfE+JcO4mtpaU5qFuapZBQVaTJjQdjtIFEXs2ZPTpUV57hxZjtnZ9Lrh4SaP1fHjmXjllWkdnvP005kYPHgKcnPJJXvyJHDmDIlhV/j5kfXr6UmHKJJ1rNfTUV/fvaAKghFhYQqMGwds2dL1+3VFnxXJnTt3YurUqR0enz17NlatWtXt853VT1IuMl5WRqup/HwS0YoKLRobs3H+/D5s2LAQRiO5aekvMAXAAyA3rLxMPAtgGYAMAJUQBAVSUpYgNHQcLrwwCZMm0d6lI0RM3qcMDGz7xWEYxjlIEs0bxcW0GO9qytq1KxO33dZR0Navz0Ry8pQOj9fVkTgeP04/R0V1dN923BMMgCBcjxEj3sapUz6orW17vihSj0eNhmIpIiNN5TBlF2p3GI00nspKciuXlAB5edU4e1aPc+f8UVdHk93YscC+fd2/njn6rEjaiquaLtfXm+oY5uaSeJaXk2AOHJiI48e/xurVspUZBWAugAebfwbIFbscwFIAZQDQvHeZjj//ORXDh1NErL1do3JhYj8/cr/05VwohulNGI00T5SUkMeou+9eTy1Jg4Gsvt9+o/lIFjFz7NixGmvX7oQk3QpgOgDTatnbG7jwQuCCCygdJDbWsYtpSaJrsncvvd/ixda/FoukGVwlkq1pXd/wzBmyOBUKQKHQoqYmGzk5+/DppwthNIoA7gGwAOSWBYAqAK+DxLIcCoWI++7bDb2+BhdfnIQpUzQIDLT6Y3WKvJr19CShdHRqCsP0dwwGmiNKS6k6V0+3Vdaty8DTT1NgoSiKePnlFW32JEtLSRyzs+k1IyI6d91KEp3z44/Ar7+Sm1cmOhoYNw646CI4zIvVHadPU9Nl3pN0AL1BJFvT2Eib3Dk5tGEup2EAWpSWZkOp9MWLL04EcC2ARQBGNz+zEsDLAJZBEBpbgoNuuSUdc+emIinJ/vuVckBPZCSNkQN6GMb+6HRk4cnzi6X5jwUFWuTkZCMhIbHFgtTpaN/wt9/IjRkT03k9aZ0O+OUX4NtvKT9SJjwcuPRS4OKLyS3ralgkHUhvE8nWlJdThNnJk7SZ7ulJrpC9ezOaXbFGADcA+DtMUbFaAP8HYA0AIwRBxJw5uxEUVINp05IwerR9w6Tr60nYw8OpFBW33WIY+9HQQHNLba392tqdPw8cOACcOmVqvtB+gVtXB2zfDuzYQV4jgNym48cDkyZRgZPetChmkXQgvVkkZRobSSyzsuhf6tStRXV1Ns6c2Yf1659p3h9YDCC++VkHATwM4OfmQCCyLB96KB1PPJFq173EpiZTI+ewMI58ZRh7IEewNjbax1NjMFClnIMH6fsaE9Nxv7CmhqzGb7+lBTBA5eauuAKYPNm1udJdlaJztkj26WIC7ohKRTfAgAG0b3nyJJCbq4FKpcHkyVMwYcLtKCnJhkJxDosXLwfwLMgNuwvAakjS0wCKIUlGLF8+D/X1atx4YzLGjrWPValU0uLg/Hn6IkZEcCcRhrGF6moSSIMBdokpqKwk6/HECRLcAQPa/r6hAfj6a2DbNtN+Y0wMcM015FJ19fe5s1J0lhYwsCdsSVqBIy3J9kgS7VGcOEErKLkijo+PfDM9B0l6EUAqAAVov/JZUJ4l/WkFQYH588mqtFdvS4OBvowBATQeThFhGMuprKT5RKGwvYWeJFF8w759tGWj0bTdezQYKBhn40Z6X4AiUq+/Hhgzpndsn5grRde6gAFbkkwbBIGCZSIigMGDTWJ57hwwYUIqhg9PQUlJNs6d+wPvvVcL4GJQusidAO4HcLzZqkyDWn0REhJqMGaM5SWr2iOKtOqtrKS80Kgo1zeXZhh3QU5nKC7uWYpHdzQ1Ab//TsE5okgi0tple+wYsHYteacA2iq5+WayHG0RR6ORXMQGg+mQza7KSi3On89CZGQSwsM1LQUEusJcKbqSkmyXdAABWCTdBkEgIYqMpI30o0cp59LDQ4PERA0uuAAwGjOwevX7kKSXAFwK2qv8F4DFkCQ9liy5BJJEJauWLEnHXXfZ5sJQKExCmZ9P43NmhwCGcUfkEm7FxSSO3S0uu6vHWl5OqRrZ2RRU19owKi8HPv6YolYBslavvx6YOtWy5siSRGJYW0v7l01N9LhCQSLv4UE/iyL9u3t3Bj7+2OQyveGGdIwalQq9nuYyuSG9j0/bBUJERFJLXIWMQiEiPDyx54O1M+xutQJnulvNYTCQSB45QgLl70/BNOXlWhw6dAhr1ggArmk+ez8o5/JYy/MVChGZmTlITLTP6qyqir4okZFcHJ1hzGE0mirJmMuBbC2K33//dZf1WHNzSQBLS4G4OJPwGY0UsbphA+07CgIJ40039dytq9fTArimhuYblYpELTyc5prWIqdSmYqVFxV1LGygUIjYvDkH/v4a1NSQJ6ysjPZjGxpo7lCraftm166MluIqCoWI2bNXtNmTZHcr0yNEkapOxMRQePdvv9G/UVEaTJumgShmYPXquyBJrwMYCxLK/wPwXwBGGI0GvPbaetx7760YM8Z2ofT3py9TQQEJpSsXEAzTGzEYSBzOn+9ZkXKh2V8q2zFGoxFPP52GyZNTEBGhwdGjwP799LzWDZDz84GVK2k+AEhQZs2i5gjdodeTAVBZScLl709VdSIjKfLV3797C/TMmawOLQWNRgNqa7MxcmTb6j8VFSSWhYU0d+TlAXFxqfi//0tBfX02IiMTXeZmlWFL0gp6gyXZnspKcsGeOEE3X3Q0VfE/fToPX389FNnZgc1n7gRwNwDqoioICjz+eDoWLEi1y8Z9XZ1pj7KrclcM059oXUXHXJGAzkrLdcYHH2RCFKfg2DESrqAgelyvB7ZuBb78kn729gZuvZXSObr6bstVtcrKSGiDgkh0o6LIYrQ0KM/aYusNDeSCPnuWjooKtDSjbz0GtiQZqwgIACZOpNXioUPkhgkK0mDcOA3GjgUyMn7Erl2jQIXUfwdFw26EJBmxbFkaQkMvQlxcDS64wLagHh8f2rPIzyeXD7fbYvo7ej0JZFkZfU/NBa90ZoG1R6EQkZ+fiKoq8iLJ+3lFRUB6OpW6BICRI4F77qHiAeYwGEi0q6pIuIcNo/nD1mj16GgNXnklvUOJvO7mFS8vev/4eFpsa7W06C8sJCEPD3dNzANbklbQGy3J1jQ20s31+++00S4nEp88WYgVKwSUlUU2n/kWqDZsAyh9xLI+dF0hV+eJiOi8ygfD9Af0etN80ZVAAp1bYIIgQBAULftzM2euwKBBqYiPJ2tUkoDMTArOaWoiEbn7buCSS8x/52TRbmggK23wYEoFaZ2jaUvz5tav0b5EnqUYDCSS2dm0AGhqonnlwgu54o5DsKdIyqvC3jz5nz9PFTdOnaKxhoQAJSVaPP30hwCeaj7rIIBbAJxueZ5CIeKXX7p2jfSExkZaEUZE0Hv35mvFMPZGp6O5orKSBKi1y9OcCHVWpPzyy1Owa1c2SkoS4e2tgUZD36XqauDtt+tx7BiZk0OHAqmp5q3HkhItTpzIgr9/EgYP1mDoUAr2aR88ZG3zZkciSXQtT5ygdJbBg8lSthYWSTPYSyTPnyeR1OtNOUEKBW1oe3jQv71FEPR6qtrTujzVnj0ZWLVqPSTpfQDhoAIEcwBsbHnevHlLMXfurYiJsU0o5TJ2slD2hoRlhnE0XQlkdyLU2gKLiNDg99+pgo68PwfQd/q112pRV+cL8gQtxJw5IzB5ckcxMxqBLVsysGHDvJYUsMWL03H33R3PtXY/0VZ6arnKHUr8/KinpLWwSJrBXiIpSfQlaN1Vu7GRXIx6vSmHSA6JVipdX+O0tJQi4U6fpo15SdLi1Kk8fPnlCJw9K/e/ehXAQgAGABTUs3hxOmbNsm0VKQtlWBgdLJRMX6apiQRS7urT3oLsqQg1NdF39vffyToMCCDB27oV2LBBgtEoAPgDwG0ADneoTCMXLMjN1eJ//2tbxcbce1ravNkeWGq5VldTdHB0tPXvaYkWWDxdzZ49Gz/88IPVg+sLtE6GVatJdCIjqb9aQgLVSoyJMUV31tRQUm91NYmpK5YlISGUJ5WcTHsRNTUajB2bjOefV2P48N+bz/orgK8BhAAAJMmIZ59Nw6lTWpveW6mk63TuHB3dxCYwjNvS1ER7aFVVHS1IoPPgHIPBgJyc7DaP1dUBu3aRBygiguaS2lrg9deBzz5Ds0C+D2AcgMMATJVp5Oefbt5B0Wg6VrHp7D0BYMCAJCjaDVoURSQkOCaZv6BA2yKQgCnNpaDAtjnHnlgskpWVlZg+fTqSkpLw0ksvIT8/3xHjcksEgSxHHx/6gkRFkWAmJJBo+vq2zUNqaHCuYHp6UrPUK6+kSDG5Fuxf/nIRZs8uhYdHE4ArAOwDMAoAffHWrNmNbdsybbpxPT0p0OncOQrzNhjs8YkYpvcgC2R1NS2cO9ty6YkIVVUBP/wAHD9Oe4Y+PpQS8cILlA/t6QnMnFkGQbgXQG3L8xQKEcHBiTh7ljxHI0YAM2YA06b1XPjkyFSx2fXV08hUa+nposGVWCySGzduRH5+Ph588EF8/PHHSEhIwIwZM/Dpp59Cp9M5YoxujUJBFmdgIBUcTkigGz8wkCyqigr6UsguWmcQHU1COWoURbmdOwdMmRKCJ54oA5AFIAHAzwBmQhAEvPvu7UhNnYYJE+Kxbl2G1e8rJyeXltL7slAyfYWmJkqGr6kxL5BA9yJUVgbs3EmRnAkJ5IXZvRv45z/pexoaCjz3HHDVVcGYMycdCgW9jkIh4pZbVqCqSoOICCAlhRolBwZaLnx33JGKPXtysH59JvbsyXFo0I6zLVdrsHlP8sCBA1i5ciXeffdd+Pn54e6778ZDDz2EpKQke43RbthrT9Je6PXkFqmupi+XTkeb815eztm3MxppE3zfPhpDXBzw3Xdr8OGHoQBmNJ/1D1CjZ7pN7BH5KncQCQ4mV5Kr92sZxhYaG8mCrK3teV5wZ+kRxcXATz9RYKBcHefTT4GvvqKfhw8H0tLaln0sK9MiPz8bRmMiIiM1GDkSGDLEfLk7W1MyHEFnEb29aU/SJpEsLCzE+++/j5UrV0Kr1eLmm29Gfn4+vv/+e7zyyit44oknrH1ph9DbRLI1crpEeTkFACkU5GaxpAixtZw7B+zdSwUIoqOBujotPvlEwq+/xjafsR7AbADUmTUtbSlSU22LfJWFMiiIe1Iy7os1AtkZ+fnkYq2pocVqQwOwYgW5VwHq9XjTTR0Xz6Wl5I0aOBAYPZq2UXoDRqPpkBWmdSaAQkHXSl4gWyLgvV4kdTodvvjiC6xcuRLffPMNLrroIsydOxd33nlny5tt2LAB9913H8rLy63/FA6gN4ukjNFIX7iqKroZDAYSy85Whvakvp7CzI8cIZdoSAiwbVsZPv7YD4AStE95HYAiABT5+vLLtnUSYaFk3BGtVousrCzExSVBFDU2C2RuLgXpNDRQ7MK5c8Brr5H71tMTuPdeqqbVGr2eKtJ4e5M4DhninAU1YErXSEhIgsEAZGdnQaNJQliYSdxkIZTFEKB/JYmO1iIqI2cDyKl05nC2SFo8LUVFRcFoNOKOO+7A3r17MWrUqA7nTJ06FYH2aLHdD1Eo6Abw86MvTVUVCUltLYmlo3o2envTFzEoiNyvWi2QkhKMqqrN+OqrCaAouj2gziJHIUlGLFyYhkmTUhAXZ51FKYoUtSevpVgomd5ORkYG5s0zpSssWpSO1NRUqwXy1Cng559pwajR0PbH66+TEAQGAo88QlZia6qqaE8/IYFyBSMibP1UPWfNmgw8+2zHAuwKhQL//W865sxJbWmb1RORlHtQyml0dXWmVDo5i0DuMOIqLLYk16xZg1tvvRVebthh1x0syc5obKQvRnk5BQiYa7FjL/Lzqf1OcTG5fs6cKcT//qdCdXUwqPDAzQB2AAAeffQTjB8fiiFDrC9fJQcwBQZSKg0LJdMb0Wq1iI/vOsfRknJuJ0+SQIoiCd2vv1L9Vb2e9iQfe8xUvBwgcSkooH8vuoiiVx3tYZL7SDY0AIWFWvzpT+YLsIuiiJycHGg01m/DGI0UmyGXn6uqMjV1ltPu6up6eZ7krFmz3FIg3RmVipLw4+Npz0Gnoyg4R0XExsQA06dTpf3cXECjicJTT9UD+B5AAICvAMyBIAh4443bcffdtkW+ys2bKypof0evt9tHYRi7kZXVdbrCunUZmDAhHrfd1vX3QZKAP/4gF6unJ32nt2wB3nyT7v1Ro4BnnmkrkE1NlLLl7w9ccQVZkI4UyKYm8mBVVJBwhYQADQ1dF2A3GAzIzrYtdUNu4qxWUyRv67xzpZL2bBsabHoLi+GKO25IYyPdvOXldAOr1Y5xRzQ20j7l77/Tl/P331fi/feVAO5qPuNvAP7Zcr5CIeKLL3ajrq7GqsLIskUZEEAWpbP2WBimJ5w+rUVSUueWJIAeVdKRJKo9umcPbZ8EBQFr1wLffUe/v/JK4Pbb2wboVFZSxOvgwcC4cY5rQSdJ5OpsaCBB8vOjucXbm7w7nVnSrbGHJdnd+BoaaOtJFNsuIizFoZYk43pUKnLPxMXRF6a6mg57L3dUKmDCBMq3amwEhgy5F//+92Rccklu8xkvgjqJkEIbjQZcd90l3a6kzSFblJWVVNaL026Z3kJTE6BQaLBoUef5hj1Jipck6vm6ezcJpFoNLF9OAikIwJ130iELpCTR1kdNDXX1uPxyxwikwUDzR3k5jSMqirxW0dE0Rnn7Q6PRID3d9PkFQWjJcRRFEStWrHCYQNL7kWCHhtomkBa/L1uS7o0k0ZeotJT+9fY29ZizJ7m5tH8i51Nu3lyODRsCQOusjQDuABVaNmFtYWS2KJnehFxJp6aGFnGFhR3TFbqrySpJFDm+Zw95ZTw9KUAnK4tEaN484OKLTe+p1wN5eZRLPGGCKW/SXhQUaJGdnYWIiCRERGjg60vC4+vbvVdKq9UiOzsbiYmU8C//7EiBtDfcdLkfIQi02vPxIWEpK6MVYesVoD2Ij6f32LWLqoHMmBGEiortyMy8DMCNALYBuB5AVctz5JW0pSLZeo8SYKFkXEd7gRQEqmDT/p7uqtFwe4EEgMWLKQjH2xt49FHgggtMr1VXR78bNAgYP97+VtPatRlYuNAUofv66+l48MHUHhcw0Wg0bQTRncTRGtiS7GM0NJiEUqmklaE9qa4md1F2Nm2mHzlSgnffDUZTkweAQwCuAlAMgHIp33xzHcaNS7Yq8tVoJNerWk0uIBZKxpl0JpDd0T4pvr1ANjYCr75Knp/AQOAvf6HUD5nSUoroHDECGDPGvilfRiNw6pQW06Z1tHgduZfYG+E9yX6MlxcJSmwsWWRlZfbd21OraW9kxAha7V5wQTiee86jeYU8CsAuAAMACJAkCQ8+ONPqyFfZoqyupvdyZn1bpn/Tuli5JYUCoqM1SE6e0qlA5uUV48UXm1BaSjEFzz1nEkh5/7GpiWIALrnEvgJZX0+emYKCzvdObY1K7cuwSPZBBIG+lLGxtMldW0uHvfDyosIDF19MK19fX/rCU0PYQfDxOQpgOOR6r7a0vxEEk1AWFrJQMo6nJ908ukMO0pEF8ttvt2DZMm/U1SkB7MfUqWsRGkrnGgxATg59j6ZNoxqt9qrd3NREC2VJIs9PcnLnBcXl/UV3QKvVIjMzE1qtc9ppsUj2YZRKWrFqNPSlKy+3Xw6ihwflal16qanl17PP0hexrs4bQCYAUySCwWDA/v27sWuX5S23BIEmKxZKxtHYUyB/+YUE8tix8/jqq2kA/EHfi6n45JPZKCvTorGR9vjl3OTY2G5evIcYjeS2ra+nxWtcHH2e+Pi2EarOiEq1JxkZGYiPj8e0adMQHx+PjAzruxL1FN6T7Cc0NZHVV1ZGlqA9I2BPnaJ9yqYmikh99dVG5OWpAFSDgnl2AhAgCAIkqWfdxztDkshl5OdHLmWl0n6fgWHsIZAA5UHu3k2W4alTwNtvG2E0KgB8CeBWAI0AgEcfzYRaPQUXXkgRrD4+9vkcDQ2mqjShoZ3HJbSOUHUXgTRX8cia/VTek2Q6oFRSlGhMjCkgpoviGRYxaBAwdSqJV1kZsHChClFRBQDUoOo81wJAS3d0a92vsuu1pob2KBsb7TN+hrGXQP7xhykP8uhR4K230CyQHwG4CbJACoIIUUzEuHHAZZfZRyDl77VeT4tIjcZ84J5Go8GUKVPcRiAB8xWPHL2fyiLZj5BFRqMhQauosJ/rMiYGmDKFVq6FhcDzz0dj2LB6AF5QKDaB6r2aMBgM2Lx5vdVCWVtL78NCydiKvQQyK8skkIcOAe+9R96Pyy8HZs+ug0JBTjtBEHHDDStw1VUajBtnn6jthgaTlyU2lsrI9bU+rUlJrtlPZXdrP8VgIKvv3Dn6ktorVaS8nHIp8/JIjFetor0ZwADgPgDvtznfFtdrZSW5jaOiHNcdhenb2Esgs7OpYbKnJ7B/P/Dxx/T4lVcCd9xBr1taqsWRI9mIiUnEVVdpOnT3sAZJor1HhYIWqIGBfU8cW5ORkYG0NFMu6ooVK5Caanm7Pqc1XXY3WCTbIkk0OZSU0GTh72+fqLrqaqrOc+oUCeWHHwI//ij/9kEAb7c539rKPCyUjC3YSyDPnKH7WxBoQbhhAz1+zTXAzTfT4wYDVa0KCwOSk+l+tRWdjsbu50dF0u21p9nbscd+KlfcYXqEnCqiVJJFWVFBm/22un/kXEqlEjh+HJg5k+rAbt8OUK1XFYDXWs6XI1/PnAm1qDC6IFCgUFUV5ZhFRzumJB/T97CXQOblkQVpNFK6x5df0uN//jNw/fX0s05HAhkXRwIZHGz7+Ovq6DOEhZFrtT+1l2tf8cfRuN2e5PLly5GQkAAvLy9MmDABe/fudfWQ3B4vLxKYsDAKiqmvt/01vb1pQhg+nCajG28Err5a/u0yAE+1nCsIAh566HarCqPLQtnYSME89hg707eR75WaGtsEMj+fBFKno39lgbztNpNA1teTQCYm0p69rQIp1zUGKA4gPLx/CaQrcCuR/Pjjj7FgwQIsWrQIBw4cwMiRI5GSkoKSkhJXD83tEUX6wkVHU3RcdbXtr6lSUdGB0aOpgfNVV5kmD+BlAM8DUDR3Krct8jUwkFbWBQW0ymaYzmhspEVbba1llXTaU1gI/PAD3Ws//gh89RU9fuedwIwZ9HNtLd2Pw4eTZ8XPz7ax63Sm4ByNhhaH1o6f6TluJZJLly7F/fffj3vvvRdDhw7F22+/DR8fH7z33nuuHlqfQE7a12hodSr3q7QFT08q0jx2LPXEmzaN9mmIFzBmzGHIlXlkrI18DQigiaSgwL4Vhpi+gb0EsqSEhLGmhoTym2/o8VmzKFAHoL3y4mJaIE6caPt+eX09vV9YGC1kef/debiNSDY1NWH//v2YPn16y2MKhQLTp0/H7t27XTiyvoevLwmlvz+tXG2t0iOKJJLjx9PrXXopNZYFgAMHhgJ4pcNzXnhhgVU1X/39KUiioMA+1jDTN2hoMHkZbBHI0lISyPJy4PvvgR076LXmzKEFoHxORQUVCBg/3rY9fjl61WBg96qrcBuRPH/+PAwGAyIiIto8HhERgaKiok6f09jYiKqqqjYH0zNUKorACw0lsbE1H1GhAEaNosLN1dXUYf2uu+TfPgnap2yLta5XtZoml4ICmmCY/k19Pe0f1tfb5qIsLyeBPHeOBDIzk17rvvuAyZPpnOJiEuRLL6X73ZZocXn/UakkgbRF3BnrcRuRtIbFixcjICCg5Yi1V2HEfoKHB61cIyJogrF1r08QaH9Grvc6ciRwzz3ybx9DUtJBAG1ngfbd3XuKnx9NUAUF5Ppi+idyb8amJttEprKSBLKwENi5k0RSEIDUVKqYA5AQA8CkScDQobYJmrz/6O/fdeUcxvG4jUiGhoZCFEUUFxe3eby4uBiRkZGdPueZZ55BZWVly3H27FlnDLVPIScpy+XsbHVhCgJw4YUklDodTSb33kuPZ2WNAqWImGYXURTh7e1rVWF0X19ydRUUkBXQfzKCGcAUOCPXFLaWmhqKXtVqSRzlnMi5c+k+liTg7FnaJ5w8mSJZbaGhgb5n8v4j1yh2LW4jkkqlEmPHjsWOHTtaHjMajdixYwcmTpzY6XNUKhX8/f3bHIx1BASQUHp40ArXVsEZPJhW3JIEJCXRipxW3mkA3gEVRBdxxRV34/rrL7EqPQSgVBSlkiwAuWUQ0/eRe5AaDLYJZF0dCWROTluBvP9+SnGSJErxUKspxSMuzrZx19bS1kZUFHlx+nL1HHfBbUQSABYsWIB33nkHq1evxvHjx/Hggw+itrYW9957r6uH1i/w9SWh9PEhobQ18nXQIBJKUQQSEmjiIaFMxfDh+Zg/fze+/XaNzekh3t50FBVRhK29CrszvZOqKhJISSLxspb6eiqxeOoUCeRPP9H9mZZGEatGI1XbCQujAv+2VNGRA3QA+o6FhPD+Y2/BreKkZs6ciXPnzuH5559HUVERRo0ahW3btnUI5mEch5cXfYmLi017JrZE2w0YQCL5009UmDktDUhPB44ciYIkCR0sPzk95Nprb7WojJ1KRZNOcTFNbqGhvErvi1RU0GLIw8O2Mm0NDVSs/MQJEsiff6ath3nzKGpVrycLMiaGFnpBQda/l9wCzsuLhLa/lJdzF7h2K2MVBgPli5WW2qeUXX4+5ZzV15Nr9O23ZYvvQwD3gAqkm7C2MLpeTyv24GAOp+9LSBK504uLaUFkS3nCpiYSyMOHKUhn924SyLQ0SumQy8wlJFDQji1TicFAQUFqNbWyU6msfy2m53A/ScbhiCJFvYaHU2CDrSkiMTEU9ODrS6/74IOypXcngLUA2pp91rpePTxoj6qsjMRYp7Nt3IzrMRrJjV5UZHtDcZ0O2LuXBPK770wC+cADJJBNTSSQSUl0v3Y1vxYUaLsMONPrSSCDgihAhwWyd8IiyViNQkH7MZGRFOBga93U6GjTxBMWBjz0kCyUMxEbewztdwfkwuiWRr6KIqUDVFaSBdvQYNu4GddhMFDeYnExLbBsqUSj1wP79gG//UYC+csvdI/ffXcp/PzoHsvLo+jsyy7rOi1j3boMTJgQbzbgrKmJPBqhofT9sUdPScYxsLuVsRm5ZVVhIX3Zbd1Tkct+lZZS6sabb8pVfzYAmAmAzD9BECAIAoxGo1XuV3ncKhVNVJyL5l7o9SSO5eW2740bDNQH8tdfqVvN3r0kkJMn78DOnX+CJBkhCArcf386Fi5MbbH6Cgq0OHMmq033moICLSZMiG8JOAPatoNraKAFZXg4BejYoz0dYxnsbmWciiCQZSbnUtbU2PZ64eFkUcpNZOfPlyfAPwP4DIAS8q1rS+SrPG6djixKrs7jPsidPMrLyX1ui0AajcDBgySS335rEsi77y5tEUgAkCQjMjLSUFpK95g5a/HMmaw2AgmYimLU15PnIjKS7m8WyN4P/4kYu+HvTy5ThcJ2wQkNJaGMiKBJ8JFHZJfUdRg4sAB33PEx2jtBrK3O4+9PgpmfT9Zr//GtuCf19abavIGBtkUpG43AoUNkQX71Ff0rirQnHhn5e4tAysj3WEGBFk89Na/TRdqAAUlQtFM/URQREZEInY6+I5zi4T6wSDJ2xc+PJgFPT9uLDoSEUIuhiAh63UcfpcIAp0+HYP/+awG09evaWp1HpaLgj5IScr8xvY/qatM+cmCgbZaYJAFHjpDluHUrWZKiSEE648YBXl5JEISOYpeQkNiltRgdrcErr6RDbFZvURTxwgsrEBGhaanB2h6tVovMzExotZbdt4zjYZFk7I6PDwmltzft+dkilMHBZFHKr/fooyRmJ096ISoqCwBliysUIm66ybbqPF5eJMbnzpFYcuRr70GSyLWan08/29pLUZKAo0cp/3HzZuDAAZMFOW4ceRREUYPHHmsrdi+/vALR0Rqz1mJCAtWku+OOVOzZk4P16zOxfXsOZs5MRXR058UNMjIyEB8fj2nTpiE+Ph4ZGZbdt4xj4cAdxmHYq38fYCowffYsiddrr5E1MXBgI6ZN2wvACxkZl7Rxj7UOlrAEOXfNz4+sWFtSChjbkVM8zp2zLMWjs6AagATy2DEqYLFpE0WziiJFU48ZA2RlaVFUlIUrr0zCpEkaFBZqkZOTjYSExDavs25dBp5+Og0Gg6FFQFsHjrUODDNXJECr1SI+vmOQT05ODjQay+5bpudw4A7TK1CpyAL087O9gXNAAFmU8fEUpPH44zRZnj6tQmbmJBiN9Z3uH7Vu3txd3pqMKFLuWl0dFbXmgB7X0dREC62SErqPeiqQXaVg/PEHCeTnn5NAengADz9MAvnllxlYvDge7703DXfdFY+PPspAdLQGyclTOiy2WluLe/bkdCqQ3t70HTAX8Z2V1bnbNjvb8r11xjGwJdkJBoMBOva1WYWnp2eLe0pGpyP3ZWWl7ftIckeG06dpInrtNbJUY2ObcPZsNIDSDs9RKBS4+eZZ+OyzNRani9TW0vjDw8n1y9GIzqOujlI8amtpkdTTAJ2uUjCqqzX4/ntg/XqyJmWBHDkSOHZMi1dfjbfZGyGXmfP1JQuyqyIBbEm6BkssSS7K1QpJklBUVISKigpXD8WtCQwMRGRkJIRm/6qnJ4W8CwJNHpZMeO3x86NgHoUCyM4GnngCWLYMOHtWiaCgEygvHw6gbRNuo9GI9etXt/n/00+nYfLklG4nP19fchsXFZFVExbGid+ORi72XVxMru+gIMtc9eaCan76KRsVFRp89BHVZPX0pKjp4cPJY1BdnWU2mrWnIik3SlarSSC7a3Ol0WiQnp6OtDST23bFihUskL0IFslWyAIZHh4OHx+flkme6RmSJKGurg4lJSUAgKhWbRFaC6Wc22atUPr4UFFphQI4eRJYsIAsyvLyEISH52HMmJXYti2ty9ewZPJTqcjiKCsjwQwP58IDjsJgoP3H0lISGD8/y19DDqppLZQKhYicnCRs2UKLK6USeOwxqp6Tl0f347XXJiE9XdHBqpODcbpDFsiAANrL7mkfyNTUVKSkpCA7OxuJiYkskL0Mdrc2YzAYcPLkSYSHhyMkJMRFI+wblJaWoqSkBIMHD+7gejUYyEIoK7NNKAFTp4bjx2lCWraMBDgkRI/S0sEAzph9rrVutOpqEvrwcNuDkZi2NDRQcI5c8NsWi711UI1CIWLGjPdw/Pg9OH2agn8ef5zqr+bmUk7upEn0N+0uGMccrQWSy8z1fixxt7JINtPQ0IAzZ84gISEB3hzOaBP19fXIycnBgAED4NVJMc3WHURsLSfW2Ajs2UPh/Eol8L//0UTr41OD+vrxkKTjUChE/PnPd2Pjxg9aJr9nnlmMiy4a1yHysWefj47gYJpguXO8bciLj5IScmn7+9tn77egQIsff8zGmTNJ2LQpBnl5FEizYAG1aMvJIZfo5ZfT37L18zqLZjWHHA0dGEgCyZ1lej+8J2kD7GK1ne6uoSjSqh2wXShVKmqA6+FBnRsef5yEsrDQD35+h3H99QcQEhKF5GQNFiz4JwoKsvHbb/vw0ktPW13z1dubhLG8nMRSLnbAWI5eT/dAaSlZX50l2ltC67SPqioNzp3T4LPPqEKPry/wl79Q39LfftPC0zMLF1yQhODgtkIYHa3p8cJJFsigILoPWCD7HmxJNiNbkuasH6bn9PRaGo1kPZw/b7tFqddTSbFDh2jP8u23aa/Jx4fy31Qq2n+Kj9di8mTzxactQZIo8tJgoOpAwcE8SVpCbS1Z/TU19ulJum5dRkupOIVCgenTP8ChQ3egpITur7/+lYTsiy8ysHXrPEiSdYskGVkgg4PpdbmJt/vAeZJ2pL6evgjOOmxtN2WOEydOIDIyEtXV1S2Pbdy4EYmJiRBFEY8//ji2bduGUaNGdYgMdBQKBVmUYWEUzWhL1o2HB/X7GzeO9rYeeAAYNIjSCF5/nSbi48eBzZvNlxOzFEEw5e6VlFCEZG2t9Z+hvyAH55w9S3+roCDbBbJjLdV4fPPNeJSU0OsvXEiu8SNHtC0CSedZ15eUBbL/wOveLqivp4oc5eXOe8+gIOCGG+xf5eWZZ57BI488AnWrulhpaWm499578eijj0KtVkOtVuNvf/sb1q5di1mzZtl3AGaQe1ICZFXYYlGIIjB2LAnmvn1Aaiqwdi3tV/7vf8DcuYBSSfU42+fCyTVfrdmjVCrp71ZdTdZraCj9n63KjrS2HuV6ufagbdrHUADfAohGYGA9nnnGGz4+5HINCLA9zUOvp0VdSAgt8lgg+zZsSXZBUxMJpLc3TXqOPry96f2amuz7OfLy8rB582bMmTOn5bGamhqUlJQgJSUF0dHRLeI5Z84cvP766/YdQDfIQhkeTpOnLZ9foQBGjaJ9Sr0euOsu4OKLaeWfng5UVmpw9dXpUChM9ThtrfkKkFXp70+Rk8XFZCVVVXFHERmdjqzt1tajvQQSAOLj5WLkFwP4AUA0gMN47LFyKJX0Nxk9Grjmmq5rrnYHC2T/g0WyB3h50arX0Ye1W6G1tbW455574Ofnh6ioKPznP//BlClT8PjjjwMAPvnkE4wcORIxMTEAgJ07d7aI4rRp0yAIAnbu3AkAuO6667Bv3z6cOnXK1stmEfYUSkGgBPHLLiORuvFGKmknScD77wOenql45JEcPPJIJlau3N1SiQew3v0mo1KRAOh05H4tLCRR6K/I5dny8kgkvbxMrcnshV4PFBRoMGHClwB2AAgB8AvuvPMQ1OpoVFSQK378eCAurmOHDrloeU/ep6qKPAXsYu0/sEOoD/Dkk0/i+++/x6ZNmxAeHo5nn30WBw4cwKhRowAAP/74I8aNG9dyfnJyMk6cOIEhQ4bgs88+Q3JyMoKbY+Dj4uIQERGBH3/8EYMGDXLq5xAEmoAAWvn7+dmWXjFkCLlud+8GrriCXLmbNwMbNwLTpmlw2WUa/PxzZpctj6z9HH5+NKmWl5PoBwdTDl1/yp+rq6N82MpKk0va3sHjTU3kWl+3Dvj116sBAPHx5UhL00ChmICGBiA5GRg6FCgspMjXyZNTsGdPjkVpHrJAhoXRweUJ+w8skm5OTU0NMjIy8MEHH+CKK64AAKxevbpN1Y7c3Nw2IqlUKhHenIMRHByMyMjINq8ZHR2N3NxcJ4y+I7JQCgKVgrNVKAcOJOvup58oqCcgAPjwQ+C772jSmzSp4x6lQqHA+fMlKCjQIjpaY7abRHd4eJAwNDSYateGhJBY92UrpKGBEuvLy8mS9Pd3zOetrwd++QVYvRr49lt6r4svBubODcK5c0FQKKhIQGJix8hXSyJadTrab2aB7J/wn9vNOXXqFJqamjBhwoSWx4KDgzFkyJCW/9fX11uU1uLt7Y26ujq7jtMSBIHEJDKSAj0aG217vZgYYOpUes2BA4G0NFNwz5YtGlx33SoIgtj83gIkScKDD87EhAnxePzxOWa7SfQULy8SS0kiF2xeHglmX2vs3NhIQTl5eZT36ONDeY+OEMiaGuCHH4A33wS++Yau7bRp9LctLKT3njqVBLJj5GvPXeoskAz/yfsBoaGhKLcgRLesrAxhcsipi5CFMiKC3Ha2CmV4ODBlCglmSAg1b/b2ptqve/fOwsMPn8Utt3wCgEQSMBVGt8d+pSDQxB0URK67s2dNYqnX2/bZXE1DA+035ubSv56e9knrMMfx41q8+eZOvPJKLXbtosf+/Gfgjjuoik5oKAmk7EwxV/C8u7QfFkgGYJF0ewYNGgRPT0/88ssvLY+Vl5fj5MmTLf8fPXo0jh071qPXa2howKlTpzB69Gi7j9VS7C2UwcEm68Lbm6qvBAWRK3T16iiI4oAO6QHtaT+59rRHpYwgUJBWYCBNwmfPkriUlto/qtmRyIUUCgs7iqM9o1bbs2JFBqZPH4HXXjPgjz98IQhGXHXVH5gwQYucHOo3OnWqqaITYCp43pruIlpZIBkZ/tP3gIYGmhAcfVgTBenn54fU1FQ8+eST+O6773DkyBHMmTOnzaSQkpKC3bt3w9AD/96ePXugUqkwceJEywfjAFq7XuvrbY8UlVttDRtG/1+wgMqUVVUBn38+BsDNXT6/9X5lV419u0OhoLEEBdH/CwvJCiosJFeik+o5WExTkylaNTeX9h1VKlqAOFIcAeDXX7X4xz/+AUrxuAJANSRpBrZtuxDPPBOPoqIMTJnSsbRddLRlEa1NTfQ3kAtdsED2bzhwpwvkiDy5RqczCAqyPFDl3//+N2pqanDddddBrVbjL3/5CyorK1t+P2PGDHh4eGD79u1ISUnp8rXWrVuHu+66Cz7mWqm7AEEwFaAuLqZ/bakc6OVFEY/e3lTG7oEHgI8+Ag4fVgD4FILwHCTpJQiCiKuvvhvbtlFh9Nb7lXJ92tau2Z72qGz/2by96WhspHutrIzGGBBALlovL9dO1DodLU5qauhobCSr0VkBSJJE/R8zMooB/AIgEkABgGsAHGo+x4hVq9LwwAMp8PHpeP3vuCMVkyendBvRKgtkRIQpgIzp33Dt1mbM1Rutr3euG0yptE+1nSlTpmDUqFFYtmwZAGD58uX44osv8PXXX5t9zvnz5zFkyBDs27cPAwYMsPq9HVUHV5JIRAoL6RrZ+tJGI3Wn//VX+v/OncD27fTzsGFFmD5dj+hoDQYM0KK0dDfmz7+925J969dnIjl5is3jamggMRIEstDkQgUqFQmUIydvo5Hu+cZGEgzZ1S2KpjE4C52OFjJr1gCbNknQ6wUAvwO4FsDZDufbcv1ZIPsP3AXEjsirfHcnLS0NFRUVqK6ublOarjU5OTl48803bRJIRyJblIJAQilJtv1tFAoqOuDjQ+22Jk0it+6HHwJHj0aioQG4/Xbg+HENRDG0W4G0tbxd63H5+NBhNJqiRo1GWkTJzYg9PU2HKFpnbRoMFDik15Mg1deTKOp09JiHB4mij4/zRaOujv4u77xDkayAgNjYPGi1l0OSKjucb0nlnPawQDLmYJHsJ3h4eOC5557r8pxx48a1yafsrchJ6YWF9H9bFzEDB9Jr7N4NJCRQx/oVK4BTp4Dly4F77wWMxo75lIIgQBAUMBoNbcrbWduCqzMUirYLNZ2OJvSSElokCAIJmSiaBM3Dg54nCCbhlCTTodcDublaZGdnITY2CaGhmpZ0FFEk0fX1Ne9KtTZv1BJKS4HMTLr+cgxaSgpw001xOHr0CCQpG4KwD8uXL2zTINma8TQ2kiBHRtL+d28XSK1Wi6ysLCQlJbXJh2YcA7tbm+FWWfbDWdeyooKE0l4u6spKslxOnSKhefddID+ffr79dqC8PANffZXWIoovv7yiZZ/L29u3RSBl2rfgcoS4SBIJp8FAlqb8r9HYcbKXRXXDhgwsWmRKrF+8OB13390zMbclKb8r5GuTkJCEs2eBTz/V4uuvx6KszBMeHsCsWVQMoqCAFjWXXEIBOpY2SG6PLJAREe4hkBkZGZg3z3T909PTkZpq+/Xvb1jibmWRbIZF0n4481pWVpJQeniQS9BWGhqA/fuBI0do/+3TT4EDB+h3l14KjB6dj6amLEyblohLL9W0WGq7dmXittumdXi9RYuW4tprb8X333/tEHGxlIICLSZMsK6fpi3P7YrWwgsIoICcNQAC4eNTi7lza6HTHYVKlYTkZA3GjrV9Pxqgv3VDAwmk7MbvzWi1WsTHd7z+OTk5bFFaCPeTZPoNAQFAdDRZUPbo5ejlRVbKxIlkjd14I3DTTTSB7toFbNwYg8DAKTh2TIOffyYrBOg8Fw8AXnhhAcaPj8OTT95vtyLqtmBtYr2tzzVH22o4IoCXAHwJIBDALtTVJeKNN6Lw1lvT8Npr8cjJyeiXAgkAWVmdX//sbOuvP9M9LJKM2+PvD0RFkTuxpsb21xNF4KKLqMyZWk05lQsW0PtotdTE+exZ4PffKRq2sLBjLl5rJElCe4eNwWDA5s3rnS6U1iTW2+O55jAJbzSA7wAsbP7NawCmAihq2QeWJCOeecb2xYWcbxsV5T4CCQBJSZ1f/8RE668/0z0skkyfQBZKgCql2IP4eGD6dArm8fICnnoKGDyYJtiVKyllJC+PhPLwYeCWW1KxZ08OFi1a2qPXf+GFBVbXg7UWSxPrbXlud9WIKKI0CcB1AA4CuBxAFYBbADwOQNfhObZarnJKV1SUY7qSOBKNRoP09LbXf8WKFexqdTC8J9kM70naD1dey5oaKjOn15Nw2oOGBuDgQdqn9PSkzhNffkmWa3g41QxVq6nc3dixQENDx7271pGw7bHHvp6l2BLw0pPndhfgc+4cRRO/+y7tAROHANwGQTgFSTI2N1Fua4Xbcq3k1JaoqI5VedwJrVaL7OxsJCYmskBaCQfumIFF0jm4+lrW1ZELtKmJ9iztgSRR1Ouvv1IJu/p6ICODquMoFMCMGcDIkeS+Gz0a2LcvA88+m9YmPWHy5BRs3rweL7ywoMPr26MIQW+hqwCfsDANTpwAtmyhfNTz5+n3l19ejTFjDkKSBkKtBkJDs5GcnIgffvgaTz/d9jpaE/BUW0v71lFRnd8TnFbRv2CRNIM1IimH1DsLhcL+pb5OnDiByZMnIysry2whgdacP38eQ4cOxYEDB6yaMFwtkgCJWEEBhfgHBNjPrVZaSpZkTg4l9H/+ualiT2wscPPNZMEOHAhER2tRWdnW4nJUhGhvoaBAiy+//AT/+MdfOvzunXcyIUlT8P77wM8/0/cqIACYOxeIi6Pcz4EDgfHjTWUI5de0Jc1D3qeOjOzcu8BpFf0PFkkzWCqSBgMFaDi7LF1srH2F8qabbsLYsWO7LSbQmr/+9a8oLy9HRobl+2W9QSRpHGRR1tWRe81eQtnYSHuQhw+ThZmfD3zwAU3GoghcdRVZlQEBFAA0ZEjbUm7r1mV0aR05I1m/PfZ4z7apHG1RKETcc08+tmyJwLlz9Ni4ccDdd5NlrlAAI0bQYc+yd9XV9NpRUbSoaQ+nVfRPWCTNYKlI6nTAmTOmaiaORq8nYR4wwH69+PLy8pCYmIgzZ84gJiamx887evQoxo4di4KCAgS3Xtb3gN4ikgAJWlERCVhAgH0LhZ89S/tpBQU0Aa9fb8qpDA+n9JGwMOprOHIkLX7k9zdnHTkqWb8r7PGenVnIMoIQhPj4H5GbOwySRPu3s2ZR1HBBAaXwjBtn6v9oLyor6XsUFUUVhDojMzMT06Z1zG/NzMzElClT7DsgptfQJ/Mk//WvfyE5ORk+Pj4IdPKuu4dH2zqZjjqsEeIpU6bg4YcfxsMPP4yAgACEhobib3/7W0uwwyeffIKRI0e2CGRtbS38/f3x6aeftnmdjRs3wtfXF9XNoaHDhg1DdHQ0NmzYYNvFczEqFU3C/v5UoacH3cJ6TGwscOWVJIBNTdT498EHyWotKQHS04Fvv6W9zG+/pfqj8h5cdLQGyclT2ghk25xB5+RT9uQ9e9Izs7McSgAYMWIrfHzOISeHBHLCBODFF+lvcv48Xbs//cm+AilJ9LdWKqnJtjmBBDitguketxHJpqYm3HrrrXjwwQddPZRex+rVq+Hh4YG9e/fitddew9KlS/Huu+8CAH788cc29Vh9fX1x++23Y+XKlW1eY+XKlbjlllva7FmOHz8eP/74o3M+hAORrYmgILIu9Hr7vbavL7Xdmj6d9tGCg4FnnqH/CwKwdy/lVe7ZQ3mVW7fSzxUVHV/LEcn63dHde/a0Z2bHHMqLAfyAw4dnoLZWRGQk8Ne/AnfdRZGtgYF0jZKT7VMpSUYWSB8fEsjuyhVyWgXTHW5T4PyFF14AAKxatcq1A+mFxMbG4r///S8EQcCQIUNw+PBh/Pe//8X999+P3NzcDkXL586di+TkZBQWFiIqKgolJSXYunUrtst9opqJjo7GwYMHnflRHIaHBwVuKBQUfKNW28+lLQiUSxkeTkJ4/Dhw2WVkNa1fTwW6v/qKUh5SUmgP7tQp4IILgKQkUzCJLDTt98faJ+v3dP+wJ+d19Z7mrMzOemb6+mrw0EPp+N//lgB4EcDtAOgaX3cdcMUVZDlWVQFjxtBerb1blhqNJJBqNf2te7q3mZqaipSUFE6rYDrFbSxJa2hsbERVVVWboy9yySWXtDQBBoCJEyciKysLBoMB9fX1HfYFx48fj2HDhmH16tUAgA8++ADx8fG4/PLL25zn7e2NOrnuWh9AFKkMWXg4BXQ0Ntr39X18qKRdSgq5YgUBuO8+4KGHaG+yogL4+GNq/XTgAInml1+StXn0KAnas8++3GWyfk8tu56e11WBgJ5YtmVlwL59wPvvA7t2pUKhOAngdgiChEsvBRYvpsUCVSUi1+qECfYXSIOBrm9gIL2PpcE/Go0GU6ZMYYFkOuA2lqQ1LF68uMUC7a+EhoaivLy8w+Nz587F8uXLsXDhQqxcuRL33ntvG6EFgLKyMoSFhTlrqE5BoSDBEkUK6DEa7d8vNDqa3uP0aeC338jN++STVJBgyxayqD76iMR60iRgx47V+Oab+yBJFDjz9NNLMGbMxR2Cenpq2VliAQLAHXektnQzaf2e5qzMuLhE5OcD2dkkkDt2UKQvnSZg2DDgttsEBASQazUkBJgyhYot2Mt6b41OR4uekBC67s4IsmP6Dy61JBcuXNhcicT88ccff1j9+s888wwqKytbjrNnO3Yy7wv88ssvbf6/Z88eJCUlQRRFjB49GseOHevwnLvvvhu5ubl4/fXXcezYMcyePbvDOUeOHMHo0aMdNm5XITdvjomhCdYe9V7b4+lJqR9XX02WkyQBgwYBzz8P3HYbRcOWlACffQZ8/fW1kKR/AIiG0WjEyy8/g6amRHh7a9A69ryne5bW7G12FkjU3spUKEQ8+ugK7NunwWuv0WdZupQWAkYjRas+8wwFL+l0lIIzbhxdgwsvdIxANjaSQIaFkZeABZKxNy69pf7yl79gzpw5XZ4zcOBAq19fpVJBZYekK3sGejjiffLy8rBgwQKkpaXhwIEDeOONN/Cf//wHAJCSkoK5c+e25OTJBAUF4aabbsKTTz6JP/3pTx3cTHV1ddi/fz9eeuklqz9Pb0YQyDUnikBxMQX0+Pvbv5annx+VqhswgPYqs7JIPC++GDh0CNiypR4VFSEAngMV9/4KRuN72Lz5FPLyNIiIMO13JiR0v2dZUKBFaem5Ts/z9vbFrl2ZPc6FNBqBlJRUxMam4MCBbFRVDcGhQ1FYs8YUpSsItL943XW0D1hSQm7PYcPoCAmxx1XsHLlQubs0S2bcE5eKZFhYWK925ykUFEbe1GTf1IGuUCotz+W75557UF9fj/Hjx0MURTz22GOYN28eAGDGjBnw8PDA9u3bkZKS0uZ5qamp+PDDD3Hfffd1eM1NmzYhLi4OkyZNsvqzuANqtcn1WlFh/1xKmeBg6kc5eDAF8mRnk3A++mgp/vGPxwA8BirwfS2Aa/HFFwbk5pKLMiaGxhkUpMHDD6fjf/9r2/hZFrzW+Y5yrVhJMkIURdx0090tTaHN5UJKElll5eV0nD1LlYUOH9bg2DENcnLQYtl6e5OreNo02l88d46u35AhdERGOla0ampIxGNi7FtRiWHa4zbOiby8PJSVlSEvLw8GgwGHDh0CACQmJsKvs1IadkAUKQCjt5el8/T0xLJly/DWW291+J2HhweeffZZLF26tINI5ufnIyQkBDfccEOH57322mt4/vnnLRuImyKnCxQXm4TS3qUBZcLC6BgyhKzK06c1uOaaq7F16zRI0kAAqfD2fhi1tb74+Wcq3+btTe7KuDggKioVjzySgrq6bMTGUiTm0aNATU3bfUhJkqBQCHjrrU+g0SS0CCRAe5RPPZWG4cNT4OenQW0tfe7z5ykQ57ffSpCVVYuSkhgUFirbjH/IEErbGDeOLLnycpOrdfBgsngdKViSRBGynp609+ugrz7DtOA2Ivn888+3RGMCaNkrc3RlDFF03ITpLNLS0lBRUYHq6mqo1WrU1dWhsLAQS5YsQVpaGpTKthPh+fPncdNNN+GOO+5w0Yidj1x0wNOTUkT8/MiqdxShoXQMHQpcdFEqxo+nFITQ0ETExvoiP58iYA8cIFew/DMABARooNGQK3bv3lIIQjaA/A77kEajEcePh+H48ZpOfmfA6tW5UCo1OH+eBLKgAMjPN8BoDG9z7oABVLR94kQS7PJyilYNCaFo3vh4x7pVTWMmMff1JUvV3gFXDNMZXJaumd5USs0SpkyZglGjRmHZsmU9fs7f//53/Otf/8Lll1+OTZs22d0Sd9drCdBEXFpK7kOVynkTcWMjiVRuLrk5a2qoh+XRoxlYt+49AFcAmApRnASDwdzatgzAWQDlAGoA1GL48GthNAo4dmwbACWAEAARACIBmMvDKATwE4BtEIRtePHFX6BQkMUpC9SgQT1L1rcXej1ZkIGBZK06cgHD9H24dqsZ+qJI9kbc/VpKEllvJSUkmmq1c/e8KivJUjt4UIsnn4yHJLXuS+mDhx46g4ICT2zY8BmAkQCSAARa9V4hIWRBR0cDHh7HsGXLtQDOtDln7txMjBs3BQMHdmxWbG1hdEue19BARerDwsj6dnfPDuN6LBFJt3G3MoyzkCNfPT1pn7K8nP7viICezggIoKO0NKuNQAKAJNWhsvIY/PwkAPe3+o0/gFhcc82bEMUQeHlFwcMjGHq9qUB/Q0MZ9PoCxMSEIzY2vCWat6GBjuJif2zdmtsm7UShEDFzZiIuuKDj57e0MLosjL//vh8vvfR0j54nB+hERVHwEwfoMM6GRZJhzODrS4W35bQGR+9TtqezZH6FQsQNNySishL44ANFKxGtgiD8gcTEgfD31zSfaxI2SQKMxmBIEnV0qamhRsRKJbl1AwOBxEQNRDEdS5e2jZ4dOrSjpWdpwQJzbbTMPU+25lUqEsgetEFlGIfAIskwXaBU0iStVFJwi15v/5Jq5pCT+dv3nkxOJjH5979Nv1MoRDz11ApceaWmJWVJp6NDjpiWLUqlkg6Vij6Ljw8JJQBMnJiKmTM7Vt9pT1cFC9o/p72gtqf98+QKOmo1FQhwQ48904dgkWSYbhBF2g9TqUxWpbNy88yVjOvud7YQHa3p9rV6WowdMN9Gq7PnyQUCwsJov5Qr6DCuhm9BhukBgkDCqFSSUJaXO8/92pVo9UTQHIE5K7ezsXQmqDLy86KiNKispAUJFwhgehMskgxjAd7eNImXltKh03Xd1Lcv01NLtjNBfeaZxRg5koq4h4VpWhYdERGc/8j0LlgkGcZCPDwoV8/bm/Ipy8up7mt/TE3oqSVrTlBra+lg9yrTW+nT/ST7C3PmzGnpmuLp6YmIiAhceeWVeO+997rcC2rPqlWrEBgY6LiB9iEEgYRRo6HI0MpK2k9jzNO604heTyXwFAq6huHhLJBM74RFso9w1VVXobCwEDk5Ofjqq68wdepUPPbYY7j22muhd1Ybk36InKKg0VA+X3m584rhuyu1tRS9GhxMtZEd0X2FYewFi2QfQaVSITIyEjExMRgzZgyeffZZbNq0CV999RVWrVoFAFi6dClGjBgBX19fxMbG4qGHHkJNczPFnTt34t5770VlZWWLVfr3v/8dALBmzRqMGzcOarUakZGRuPPOO1FSUuKiT9r7UCjImoyLM1mVtbWuHlXvQ6+nRYRsPcqpNQzTm2GRdABarRaZmZnQarUuHce0adMwcuRIfP755wAAhUKB119/HUePHsXq1avx3Xff4amnngIAJCcnY9myZfD390dhYSEKCwvx17/+FQCg0+nw4osv4rfffsPGjRuRk5PTbR/Q/ohsVcbGkhCUlVGbtf6O3IKrpsZkPXL0KuMu8C6AncnIyMC8eaZSXenp6UhNNV+qy9FccMEF+P333wEAjz/+eMvjCQkJ+Oc//4kHHngAb775JpRKJQICAiAIAiIjI9u8Rut+kwMHDsTrr7+Oiy++GDU1NQ5rU+auKBQkAD4+JJJlZbRX6efXPwN76utNnz8khP5lcWTcCbYk7YhWq20RSIBKbqWlpbnUopQkCULzrLR9+3ZcccUViImJgVqtxqxZs1BaWoq6urouX2P//v247rrrEBcXB7VajcmTJwOgHp9M53h6UjpDXBwJQ1UVWVL9pZ1AUxMtEOS6q7Gxzi8UzzD2gEXSjmRldV6qKzs720UjAo4fP44BAwYgJycH1157LS666CJ89tln2L9/P5YvXw4AaOrCJ1hbW4uUlBT4+/tj7dq1+PXXX7Fhw4Zun8cQvr6UVxkbS8JZXk4dLfqqWOr1VJFIrpoTF0cWZH+0opm+Abtb7UhSUueluhITO5bqcgbfffcdDh8+jCeeeAL79++H0WjEf/7zHyiaq15/8sknbc5XKpUwtAvN/OOPP1BaWoolS5YgNjYWALBv3z7nfIA+gkJBEZw+PrQ3V1ZGYunlRbmWfcG60ukoWEmuTBQczEUBmL4BW5J2RKPRID09HWLzslkURaxYsQIajePLhjU2NqKoqAj5+fk4cOAAXnrpJdxwww249tprcc899yAxMRE6nQ5vvPEGTp8+jTVr1uDtt99u8xoJCQmoqanBjh07cP78edTV1SEuLg5KpbLleV988QVefPFFh3+evoiHB/VijI8n61IQSCzldlDuSFMTWY51dRTZGx9PvSlZIJm+AouknUlNTUVOTg4yMzORk5PjtKCdbdu2ISoqCgkJCbjqqquQmZmJ119/HZs2bYIoihg5ciSWLl2Kl19+GcOHD8fatWuxePHiNq+RnJyMBx54ADNnzkRYWBheeeUVhIWFYdWqVVi/fj2GDh2KJUuW4NVXX3XKZ+qrtBbL2FhKg6ispEOnc/XoukeSKBhHjt6VP0tUFFnLfcEyZhgZQZL66u5IR7rqRt3Q0IAzZ85gwIAB8OLePDbB19IyjEZyVcrBPToduWK9vJzX6LknNDbSXqPBQJZiYCAFJalUrh4Zw1hGV1rQHt6TZBgXo1BQ5KdaTUJUV0cuzKoqElCVigTT2cEvkkSWYmMjBeQolbTfqFaTSHIZOaY/wLc5w/QiVCo6AgLIaquvp2Cf2loSKlE0NU22t2hKkqlRs05H/1cqSRT9/EgYuUIO099gkWSYXohCQft7Pj4UKdrYaLIy6+pMogmQRSeK9BxRpEMQTIeMJNFhMJCFajTSz3o9/SwIJgEOCjK5fNliZPozfPszTC9HEEyCFRBAgtba4mtoIAE1GMg9ajSaBLF1xIEsmrKYCgJZh97elMPp4UEC6enpus/KML0NFsl29KM4JofB19CxKBQmt2xrZMuwtUjKFiLQUSQVCo5EZZjuYJFsxrN5+VxXVwdvTvKyCbnMnSebJE5Foehd0bAM0xdgkWxGFEUEBga2tIDy8fFpqXnK9AxJklBXV4eSkhIEBga2FFVgGIZxV1gkWyF3v+BeibYRGBjYoZMIwzCMO8Ii2QpBEBAVFYXw8HDo3KH0SS/E09OTLUiGYfoMLJKdIIoiT/QMwzAM125lGIZhGHOwSDIMwzCMGVgkGYZhGMYM/WpPUk5yr6qqcvFIGIZhGFcha0BPCp/0K5Gsrq4GAMTGxrp4JAzDMIyrqa6uRkBAQJfn9Kt+kkajEQUFBVCr1VYXCqiqqkJsbCzOnj3bbR+y/gZfm87h62Ievjadw9fFPPa4NpIkobq6GtHR0VB0U6aqX1mSCoUCGo3GLq/l7+/PN68Z+Np0Dl8X8/C16Ry+Luax9dp0Z0HKcOAOwzAMw5iBRZJhGIZhzMAiaSEqlQqLFi2Cqn2fIoavjRn4upiHr03n8HUxj7OvTb8K3GEYhmEYS2BLkmEYhmHMwCLJMAzDMGZgkWQYhmEYM7BIMgzDMIwZWCQ7Yfny5UhISICXlxcmTJiAvXv3dnn++vXrccEFF8DLywsjRozA1q1bnTRS52PJtXnnnXcwadIkBAUFISgoCNOnT+/2Wrorlt4zMh999BEEQcCNN97o2AG6EEuvTUVFBebPn4+oqCioVCoMHjy4T36nLL0uy5Ytw5AhQ+Dt7Y3Y2Fg88cQTaGhocNJoncMPP/yA6667DtHR0RAEARs3buz2OTt37sSYMWOgUqmQmJiIVatW2XdQEtOGjz76SFIqldJ7770nHT16VLr//vulwMBAqbi4uNPzd+3aJYmiKL3yyivSsWPHpP/7v/+TPD09pcOHDzt55I7H0mtz5513SsuXL5cOHjwoHT9+XJozZ44UEBAgabVaJ4/csVh6XWTOnDkjxcTESJMmTZJuuOEG5wzWyVh6bRobG6Vx48ZJV199tfTTTz9JZ86ckXbu3CkdOnTIySN3LJZel7Vr10oqlUpau3atdObMGenrr7+WoqKipCeeeMLJI3csW7dulZ577jnp888/lwBIGzZs6PL806dPSz4+PtKCBQukY8eOSW+88YYkiqK0bds2u42JRbId48ePl+bPn9/yf4PBIEVHR0uLFy/u9PzbbrtNuuaaa9o8NmHCBCktLc2h43QFll6b9uj1ekmtVkurV6921BBdgjXXRa/XS8nJydK7774rzZ49u8+KpKXX5q233pIGDhwoNTU1OWuILsHS6zJ//nxp2rRpbR5bsGCBdOmllzp0nK6kJyL51FNPScOGDWvz2MyZM6WUlBS7jYPdra1oamrC/v37MX369JbHFAoFpk+fjt27d3f6nN27d7c5HwBSUlLMnu+uWHNt2lNXVwedTofg4GBHDdPpWHtd/vGPfyA8PBypqanOGKZLsObafPHFF5g4cSLmz5+PiIgIDB8+HC+99BIMBoOzhu1wrLkuycnJ2L9/f4tL9vTp09i6dSuuvvpqp4y5t+KM+bdfFTjvjvPnz8NgMCAiIqLN4xEREfjjjz86fU5RUVGn5xcVFTlsnK7AmmvTnqeffhrR0dEdbmp3xprr8tNPPyEjIwOHDh1ywghdhzXX5vTp0/juu+9w1113YevWrcjOzsZDDz0EnU6HRYsWOWPYDsea63LnnXfi/PnzuOyyyyBJEvR6PR544AE8++yzzhhyr8Xc/FtVVYX6+np4e3vb/B5sSTJOYcmSJfjoo4+wYcMGeHl5uXo4LqO6uhqzZs3CO++8g9DQUFcPp9dhNBoRHh6O9PR0jB07FjNnzsRzzz2Ht99+29VDcyk7d+7ESy+9hDfffBMHDhzA559/ji1btuDFF1909dD6PGxJtiI0NBSiKKK4uLjN48XFxYiMjOz0OZGRkRad765Yc21kXn31VSxZsgTbt2/HRRdd5MhhOh1Lr8upU6eQk5OD6667ruUxo9EIAPDw8MCJEycwaNAgxw7aSVhzz0RFRcHT0xOiKLY8duGFF6KoqAhNTU1QKpUOHbMzsOa6/O1vf8OsWbMwd+5cAMCIESNQW1uLefPm4bnnnuu2J2Jfxdz86+/vbxcrEmBLsg1KpRJjx47Fjh07Wh4zGo3YsWMHJk6c2OlzJk6c2OZ8APj222/Nnu+uWHNtAOCVV17Biy++iG3btmHcuHHOGKpTsfS6XHDBBTh8+DAOHTrUclx//fWYOnUqDh06hNjYWGcO36FYc89ceumlyM7Oblk4AMDJkycRFRXVJwQSsO661NXVdRBCeSEh9ePy206Zf+0WAtRH+OijjySVSiWtWrVKOnbsmDRv3jwpMDBQKioqkiRJkmbNmiUtXLiw5fxdu3ZJHh4e0quvviodP35cWrRoUZ9OAbHk2ixZskRSKpXSp59+KhUWFrYc1dXVrvoIDsHS69Kevhzdaum1ycvLk9RqtfTwww9LJ06ckDZv3iyFh4dL//znP131ERyCpddl0aJFklqtltatWyedPn1a+uabb6RBgwZJt912m6s+gkOorq6WDh48KB08eFACIC1dulQ6ePCglJubK0mSJC1cuFCaNWtWy/lyCsiTTz4pHT9+XFq+fDmngDiDN954Q4qLi5OUSqU0fvx4ac+ePS2/mzx5sjR79uw253/yySfS4MGDJaVSKQ0bNkzasmWLk0fsPCy5NvHx8RKADseiRYucP3AHY+k905q+LJKSZPm1+fnnn6UJEyZIKpVKGjhwoPSvf/1L0uv1Th6147Hkuuh0Ounvf/+7NGjQIMnLy0uKjY2VHnroIam8vNz5A3cgmZmZnc4Z8rWYPXu2NHny5A7PGTVqlKRUKqWBAwdKK1eutOuYuFUWwzAMw5iB9yQZhmEYxgwskgzDMAxjBhZJhmEYhjEDiyTDMAzDmIFFkmEYhmHMwCLJMAzDMGZgkWQYhmEYM7BIMgzDMIwZWCQZhmEYxgwskgzDMAxjBhZJhukHnDt3DpGRkXjppZdaHvv555+hVCo7dFFgGMYE125lmH7C1q1bceONN+Lnn3/GkCFDMGrUKNxwww1YunSpq4fGML0WFkmG6UfMnz8f27dvx7hx43D48GH8+uuvUKlUrh4Ww/RaWCQZph9RX1+P4cOH4+zZs9i/fz9GjBjh6iExTK+G9yQZph9x6tQpFBQUwGg0Iicnx9XDYZheD1uSDNNPaGpqwvjx4zFq1CgMGTIEy5Ytw+HDhxEeHu7qoTFMr4VFkmH6CU8++SQ+/fRT/Pbbb/Dz88PkyZMREBCAzZs3u3poDNNrYXcrw/QDdu7ciWXLlmHNmjXw9/eHQqHAmjVr8OOPP+Ktt95y9fAYptfCliTDMAzDmIEtSYZhGIYxA4skwzAMw5iBRZJhGIZhzMAiyTAMwzBmYJFkGIZhGDOwSDIMwzCMGVgkGYZhGMYMLJIMwzAMYwYWSYZhGIYxA4skwzAMw5iBRZJhGIZhzMAiyTAMwzBm+H9RjMD/T00qIwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "train_and_test_approximate_qep(qpytorch.mlls.VariationalELBO)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Objective Funtion 2) The Predictive Log Likelihood\n", "\n", "The **predictive log likelihood** is an alternative to the variational ELBO that was proposed in [Jankowiak et al., 2020](https://arxiv.org/abs/1910.07123).\n", "It typically produces predictive variances than the `qpytorch.mlls.VariationalELBO` objective.\n", "\n", "$$\n", "\\begin{align}\n", "\\mathcal{L}_\\text{PLL} &= \\mathbb{E}_{p_\\text{data}( y, \\mathbf x )} \\left[ \\log p( y \\! \\mid \\! \\mathbf x) \\right] - \\beta \\: \\text{KL} \\left[ q( \\mathbf u) \\Vert p( \\mathbf u) \\right] \\\\ \n", " &\\approx \\sum_{i=1}^N \\log \\mathbb{E}_{q(\\mathbf u)} \\left[ \\int p( y_i \\! \\mid \\! f_i) p(f_i \\! \\mid \\! \\mathbf u) \\: d f_i \\right] - \\beta \\: \\text{KL} \\left[ q( \\mathbf u) \\Vert p( \\mathbf u) \\right] \n", "\\end{align}\n", "$$\n", "\n", "Note that this objective is *very similar* to the variational ELBO.\n", "The only difference is that the $\\log$ occurs *outside* the expectation $\\mathbb E_{q(\\mathbf u)}$.\n", "This difference results in very different predictive performance.\n", "\n", "### How to use the predictive log likelihood in QPyTorch\n", "\n", "In QPyTorch, this objective function is available as [qpytorch.mlls.PredictiveLogLikelihood](https://qepytorch.readthedocs.io/en/stable/marginal_log_likelihoods.html#predictiveloglikelihood)." ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAckAAAEmCAYAAAAJLWsmAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAUpRJREFUeJztnXl4U3Xa978naZPuG91XKAWq7Lutyq5Fx23mkdFxREAUVHTcR5Rx0EdHGHQYR8aNGQZ4GBQRBXwB2UFUUNllKbWlG6EUWgLd1+S8f9ycpEuSJmm2k9yf6zpX2vQk+eU0Od9z74IoiiIYhmEYhumEwt0LYBiGYRhPhUWSYRiGYczAIskwDMMwZmCRZBiGYRgzsEgyDMMwjBlYJBmGYRjGDCySDMMwDGMGFkmGYRiGMYOfuxfgSvR6PcrKyhAaGgpBENy9HIZhGMYNiKKImpoaJCYmQqGwbCv6lEiWlZUhJSXF3ctgGIZhPIBz584hOTnZ4j4+JZKhoaEA6MCEhYW5eTUMwzCMO6iurkZKSopBEyzhUyIpuVjDwsJYJBmGYXwca8JunLjDMAzDMGZgkWQYhmEYM7BIMgzDMIwZWCQZhmEYxgwskgzDMIxL0Wg02LNnDzQajbuX0iUskgzDMIzLWLZsGdLS0jBhwgSkpaVh2bJl7l6SRQRRFEV3L8JVVFdXIzw8HFVVVVwCwjAM42I0Gg3S0tKg1+sN9ymVShQXF3dZ1O9IbNECtiQZhmEYl5Cfn99OIAFAp9OhoKDATSvqGhZJhmEYxiX06dOnU69UpVKJjIwMN62oa3yq4w7DMAzjPpKTk7F06VLMnj0bOp0OSqUSH3/8sUlXq14P1NUBNTW0NTbS1tAA9OgBDBzomjWzSDIMwzAuY+bMmcjJyUFBQQEyMjKQlJSMujqgogK4cgXQaoHycvq9rg5oaqJNpwMEgYRy5EgWSYZhGMYLEUUgKCgZ0dHJyMsDtm8HLl4EamtJDAUBUKuBwEAgIACIjKTflUp6fGEhPYerYJFkGIZhnEpjI6DRAKWlwNmzZClWV5PYBQcDISFASgqJoaeN+mWRZBiGYRxObS1QVESiWFAAXL4MtLSQKIaHA0lJQBfzjj0CFkk7OHGC/OVRUfTPDg8HwsIAPz6aDMP4MI2N5A795RfaKivp/shIoGdPshTlBp/W7eD0aeC778hf7u9PvnPpyiguzriFhMjjSolhGMZeRJFcqb/8Ahw/TvFFUaQM1L595W88yHz57iM8HOjdm9wHdXXkX79wAWhtpQ9FaCiQmAikppKvPTERCAryPH87wzCMPTQ0kDAeO0Yu1bo6QKHQQK/PR1JSH0RFua6DjjNhkewGggCoVLRFRhrvb24m0SwsBE6dImszMhLo1Yu29HQgIoIFk2EY+XH5Mp3XjhwBysrIKIiPB86dW4aVK2dBFPUQBAWmT1+KMWNmunu53YZF0gmoVEB0NG0ApTVfuQIcOgT89JPRCu3Xj25DQ1kwGYbxXESRBPHYMdquXKE8jIwMOt9ptRqDQNL+eqxcORsDBuTI3qJkkXQBajVdacXHU0GsVku++8OHSUj79weuu45cs/7+7l4twzAMIYpUtnHwIFmPNTVATAyds9pe2F+8mG8QSAm9XodLlwpYJBnbUCrpQxYTQ/HLykrgm2+AAwdIJIcMIcHkISUMw7iLtuJ44gTFG6UcC1Ner7i4PhAERTuhVCiUiI313J6s1sIi6UYkX358PH0Iz5+nAHhsLInlwIH0M7tiGYZxFefP00V7W3Hs1cvyY6KikjF9+lKsXDkber0OCoUS06Z9LHsrEmCR9BiCgymhp7WVUqi//hr44QcSy2HDgIQEFkuGYZxHZSXw44+UkFNdTeecrsSxLWPGzMSAATm4dKkAsbEZXiGQAIukx+HnR/WWiYnUsGDvXvrQDh5MTX1ZLBmGcSR1dZQfsX8/cOkSebauv96+80xUVLLXiKMEi6SHIgjkao2JoZTrb78l98ewYSSW0dEslgzD2E9rK3DyJJ1bSkupTG3gQD6vdEQ2/WAWLFiAkSNHIjQ0FLGxsbjnnnuQl5fn7mU5HUEwZsCqVMCuXcC//gXs2UOZZgzDMLZSXAysXg18+il5rPr2Je8VC2RnZCOS33zzDebMmYMffvgBO3bsQEtLC2699VbU1dW5e2kuQbIspdTrLVuA5cvJTdLc7O7VMQwjB6qrKd9hxQoq6UhNpVwILj0zj2zcrVu3bm33+4oVKxAbG4vDhw9jzJgxblqV61EoKC4ZE0Muks8/JzfsuHFAWhpfCTIM0xmdjs4Te/ZQ9mp8PDUcZ7pGNiLZkaqqKgBAVFSU2X2amprQ1NRk+L26utrp63IVfn50BVhXRw3XS0qAG24AsrK4xpJhGCPl5SSOx49TY5Prr+fBC7Ygy0Ol1+vxzDPP4MYbb8SAAQPM7rdgwQKEh4cbtpSUFBeu0jUEB1PzgcBAYMcOYOVK4OefAb2+68cyDOO9NDdTxuqKFRSWSU6mkg4WSNuQ5eGaM2cOTp48iTVr1ljc7+WXX0ZVVZVhO3funItW6FqkeGVmJqVwf/YZ8NVX1P6OYRjfQ6OhpJz16ymLtX9/Gt3H2I7s3K1PPvkkNm3ahH379iE52XI9jlqthlqOUz7txM+PGqZfuUJp3UVFwIQJwIAB1A6PYRjvprmZhih88w1w9SpZjkFB7l6VvJGNSIqiiKeeegrr16/H3r170cuWVhA+RmQkxSVLS4G1a0ksx4+n6SMMw3gn588DO3dSgk6PHvY3BGDaIxuRnDNnDj755BNs3LgRoaGhKC8vBwCEh4cjMDDQzavzPJRKuoq8ehX47jtyv9xyC9CnD8ckGMabaG2lRuTffEONR9LT2Xp0JLI5XX744YeoqqrCuHHjkJCQYNg+++wzdy/No4mIoFjlhQvAJ5/QlWZDg7tXxTCMI6isBL74AtiwAWhpIeuRBdKxyMaSFEXR3UuQLX5+ZEFWVFAG7PnzQE4OddhgGEZ+iCK1lNuxg4Yh9+zJiTnOQjYiyXSfmBj6Ip0+TYI5aRJNGWH3K8PIh7o6Gnywfz99d7nu0bmwSPoYgYH0pSopITdNWRkl9QQHu3tlDMN0RWkpWY+nT9O0IAu9VBgHwSLpgwgCuWe0WroivXSJ3K9JSe5eGcMwptDpgEOHgN27qcSrXz8aeMA4HxZJHyYqiizI3FzKirv1VhqVw64bhvEcqqtp+s+PP1JSznXXcWmHK2GR9HHUavrSlZaS+7WiArj5ZrqfYRj3UlICbN0K5OfTxA7uy+x6WCQZKBTkfq2oALZto9ucHI53MIy7kNyrO3eSJdmvH4+zchcskoyBmBhy5xw+THGP224j8WTXDsO4jro6EscDB+j7mJnJ30F3wtEnph3BwfSlPHeOGiQfOcITRRjGVZSVAWvWAPv20dzY1FQWSHfDliTTCT8/oG9filNu2EBdPcaOBQIC3L0yhvFORJF6rm7bRtnmfftyXoCnwCLJmEQQgLQ0ik/u2kXlIhynZBjH09xMfVe//Za+d9yY3LNgkWQs0jZOWVVFccq0NHevimG8gytXKHv1yBGaCRsT4+4VMR3hmCTTJcHBlF1XVEQDnU+c4Dglw3SXoiIaOnDwICXIsUB6JmxJMlbh708JPUVFwLp15H7Nzua0dIaxFVEky3HbNirv6N+fh6J7MmxJMlYjCMZZdV9/DWzaBNTWuntVDCMfmpqA7dupcUdrK3loWCA9G7YkGZuJi6NM1/37aajzr35F8RSGYcxz5QpdXB45QuUdPXq4e0WMNbBIMnYRHk4p6qdPAzU1JJTp6ZyVxzCmKC4GNm8GCgvpe8JTd+QDu1sZuwkIoDjlhQvA2rWUAavTuXtVDOM5iCJw9Cg1CDh3jso7PFEgtVoNcnP3QKvVyPo1nAFbkky3UCqp8PncOWDjRnIp3XwzNx5gmJYWqn/cs4cS3Pr1c62nRavV4OLFfMTF9UFUVLLZ/fbtW4YVK2ZBFPUQBAWmT1+KMWNmOnQtrngNZ8GWJNNtBME4oWDHDuCrryhWyTC+SnU1davatg2IiHB9D+R9+5bhhRfSsGjRBLzwQhr27Vtmcj+tVmMQLwAQRT1WrpztUGvPFa/hTFgkGYcREwOkpAA//UTuV408vgMM41DKyqie+MABarzh6vpHW0Tp4sV8w34Ser0Oly4VOGw9rngNZ8IiyTiU0FByv549SyeK06e58QDjG4gifd4/+QT45ReK14eEuH4dtohSXFwfCEJ7GVAolIiNzXDYelzxGs6ERZJxOCoVxV+uXqV6sP37KT7DMN6KTkef87VrqX3j9dfT98Ad2CJKUVHJmD59KRQKpWG/adM+thjDtBVXvIYz4cQdxikoFJTqfuECpb5XVgITJ5KlyTDeRH09zX/cv59KoxIS3LseSZRWrpwNvV7XpSiNGTMTAwbk4NKlAsTGZjhFvFzxGs6CRZJxGoIAJCaSRfndd5T5Onmy+08iDOMoLl+mi8Djxyl5LTzc3SsibBWlqKhkpwuXK17DGbBIMk4nIoIaD+TmkmDm5FC8RsHOfkbGFBVRa8aSEqBPH88re5KrKHkafJpiXEJgIMUpL1+mBunffUd9LBlGbogiNc749FPg/Hnguus8TyAZxyErkdy3bx/uvPNOJCYmQhAEbNiwwd1LYmxAqQR69wb8/IAtW+gqnOspGTnR0kJDyL/8kn7mBuXej6xEsq6uDoMHD8b777/vtjVoNBqcOrUH1dVUcyTXVkvuQopTJiQAP/5IZSJFRVwmwng+VVXA+vXGBgFpadyr2BeQVUzytttuw2233ea211+2bBlmzZoFvZ5aK2VnT8X+/avatVoaMCDH0AoKQLu2UNa2ifIFwsPJBVtYSGnz48cDQ4fyfErGMzl/njwfeXmUte2O+kdfxd3nTVmJpK00NTWhqU3gq7q62u7n0mg0BoEEqIvF99+vNPxdFPVYseJRAMK1Ql4BggCIomhSUKdMWYiePUf4tGBK9ZSlpdTK7uJFYMwYz8kQZBipQcCWLcClSxR/5As512Gq52tysmt7vnq1SC5YsACvv/66Q54rPz/fIJDmEEURgCj9BlGU7u8sqGvX/hEATFqgviSagkBuK60W2LcPKC8HbrmF0uk5+5VxJ62twA8/UA2kTkcNAnzRveouS85ce72nnsoB4Lp1eLVIvvzyy3juuecMv1dXVyMlJcWu5+rTpw8UCkWXQmkrHS1QX7Uyo6JohNDZs9R4YNw4YNgwKh1hGFdTVwds304iGREhn9peRwuaO6d3mGuvp9UWgEXSQajVaqgddJZNTk7G0qVLMXv2bOh0OgiCEtnZD+LAgf9Cr9ddawMlXrMmbaOtBerLVqZaTe5XjYYKtM+fJ7Hs0cM3r+AZ93DxIrlXT50yTreRA44WNHOW3IABOS45D0nt9doKpUKhRFSUa3u+erVIOpqZM2ciJycHS5cW4OrVDAwblozf/OZNQ1eLkye3GVpBCYIAyTpUKJTIyjIKqrX4opUpCDRJpKqKpolcuEBJPf36cSyIcS6iSI3JN2+mSR59+sjHk+EMQbPUKN0V5x5z7fXCwlx73pOVSNbW1qKgwNjJvqioCMeOHUNUVBRSU1Ndsobk5GT075+MEyfo97ZdLTq2ggLQri2UJKhFRYewbt1cqyxQX7Uyw8PJ/VpcTM0HbrgByMripB7GOeh0VJK0YwfQ3EzxRznFxJ0haOYsOVdO7zDVXq+w0GUvD0BmInno0CGMHz/e8LsUb5w2bRpWrFjhplW1p2MrqI4/R0UlIzNzHEaPvt+kBWotpqxMOU37tgY/PyAjA6ioAPbupfZf48YZGxIwjCOoq6PknAMHyLWalubuFdmOMwTN1kbpzsLd7fUE0Z4gmkyprq5GeHg4qqqqENaNQMNnnwEnTtAJ3FFotRqbrcyOKBRKzJt3AE1NtV5nWTY1kUgGBACjR5NlGRbGsUqme1y4AHz9tTH+KGdPxb59yzoJmiMumqVzkyund1hKQCospAvladPsf35btIBF0g6cIZJtafuhtNXKlK4mvTF+KYpUq3blCtCrF9VUZmRwrJKxHVGkhvtff01lRxkZ8ok/WsIdguZoukpAYpF0InIRyY5018r0NldsYyM1IFCrqUwkO5tKSNiqZKyhuZlmP+7ZQ+0Qe/WSV/zRm9FqNXjhhbRObuO33y42iL6rRZIjOzLAmlhmx3hEW6RMt+TkQV7hig0IoMxDrZamiRQXk1Befz21umMYc1y9CmzdChw5QqVFcXHuXpH3Yk/Nprszak3BImkjX35JvUbPnaNkkoYGsmyamugKtamJOnW0tlLGnE5Hrh3J4BMEumpVKmlTqchdqFLRyT8wkLagIOoPGRpKW3g4FTVHRJjOplWpgvHmmzeYFUq9Xmf4uzdYloJAJ7mwMPpfbNgA5OdTBmxSEif2MO0RRWqk//XXdNurF2VPeyIdxcXdvUvtwd6aTU/IqO0In0psZO1aEkp3EhwMREdLWzJiY5MRHw/ce+9qrFs3FaLYavJxbWuoVqyYhYCAUGRkZMvmi2cKf39qOF1VBRw7RifAkSOB4cPpooJdsExrK3DoEGWw1tXRwG9PvYjqKC6mhih4+sVtd2o2PSWjti0ck7SR//wHWLOGvmyJiWT1BQRQfCwggCxCPz/jplAYrUeAYiB6vdHKbG42bo2NZJk2NNDz19bSVlNDInDlCs2ws4RKpUdUVC0CAkpQUrICongUwHEAWpP7y+WLZw16PSVhVFcDycmUBdu/P1nljG9SXU3i+NNP5JlJSvLcCydT8biOdIzPeSK5uXuwaNGETve/9NIeZGaOs+o5LCUgcUzSw3n4YbLkXJm4IyGKQH09xeIuX6YepxUVlPFZXk4/NzcrUF4eBmAggL+1efRZAIcA/AjgAIAjAJq9Kl6pUNCFS3Q0tbbbuJEyGLOyKL3fG7IXGesQRSoZ2roVKCiQR3s5U/G4jrgqPtcdF68jXKburo1sC4ukjBAEEujgYGrd1pHWVhJKjYb6nmo0lAVaUQEAva9t913buwnAYQDfQK/fizfemAigxissS5WKXLA1NSSSJSXAwIHAiBFAfLznutoYx9DaSpbj7t30GfDEloamRMiUuHSko9jYImbW7tvdHrCe6DLtDuxutQNXl4B0l9paEorTp6tw+nQtiov9AcR22KsVwE8AtgPYhVmz/oB+/bJk+8GWEEWyvCsqgMhIGuw8dCiVjCiV7l4d42iuXKHWckeOeK571ZIIdWwI0Lbnc8cGAbaImbX7WlOCYS221GzaIvZcJ+lEfFUkO/LNN8uwcuVfIYpZAMYDGAugV4e9qgDswE03xWDKlLEe76rqCileWVNDaf/DhgGDBpELjmvk5I/UnHz7dioJSkvzTPeqNSLUUVxMiY0tYmbLvo6IJ9qKrZYrxyQZpzN27EwMHNi2dCQDopgCYBKAWwFMBNADwL347jvgu+9EJCZWY9gwETfdFCHL2jIpXtnSQq3Itm6lC53hw6m+MjTU8ywOxjoaGoDvv6ea2ZYW4LrrPNelbk0doKn+zx3FzJZ6Qlv2dXUJhrvHcVmDh36UGGfT9otnjB8sA7AMgALASAC/urYNQ1lZOMrKgE2bKHN05Ehg1CiK8ckJf39K4mhsJLH8f/+PxHLIECoNCAlhsZQLokhx9507KfYcE0ObJ+MoEbLleWzZ19XxRE9sHtARFknG0JSgoOAAPvro/msf2h+vbX8GkAjgbgC/BjAeGo0fNBpg/XpKIJIajvfo4cY3YSMBAVRQXl9PyU0lJcDPP1O8sm9fSo5isfRcmpup9nHvXuqik55O/1NPx1EiZMvz2PqapsZTOQtPbB7QEY5J2oHcY5KWaJs4YDrTLhLAXQDugyDkQBSNAb1+/ajcYuRI+dUm1tWRZSkIJJ6SWAYFsVh6GuXllLl67BjFHV2dnOOIDjiOakRua3KMo15Tev8Aun0sbJ1ewok7ToRF0jqkL1NXre4EIQa/+tWP+PnnCJw7FwFRpDOVvz/F+m66ieJDckqMqa2lk7BCAfTsCQweTOLPlqX7aW6mrNV9+yhbuWdP17eW6255hNxp+/4BAYJAg+G7eyxsEXAWSSfCImk7ba/yTGG0NlMxYsTnKCsbhbIy49979ABuvpm2qCjXrNkRSGIpCEax7NuXYpZyEn1vQBTJyt+7l1zigYHk5nf1/8HazFS59Vm1lq46ArmqGxBntzIehel4pRHj76U4ciQbr7xyACUlAs6ezcSxYyG4fJmaj2/cSCUX48bRracLTUgIXQTV1lI/2OJiSvgZOJCs49BQz38P3kBjI3DwII22qqx0j/Uo0VWSibdbmV11BPK0hBtHwSLJdElUVDJGjZqCxsZqi/FKvV6Hv/zFOGnkwQeXITBwOvbtA86cAY4fp61HDxLLm2/2/Enwkli2TfA5fJgsy/79KSbGTQkcj15PFybffEOZq2FhVKrjTpe3pSQTOZQydJeuOgJ5WsKNo+BrYcZqxoyZibffLsZLL+3Bn/70w7XBz+1pe5JYvfoR9OunwUsvAQsWAJMnkxVw+TLwxRfA888DS5eS+8TTCQoiF09iIvXK3bIF+L//o5N4RUXXjecZ69FqgW3bgFWr6OKqVy8qO3J3TFjKElUo6KqobZaoJSvTE9FqNcjN3QOtVmP1Yzq+f0EQDOcAubeeswTHJO3Al2KSlug6Exa4//7FGDlyiuHL09xM7rPdu9uLY3o6MGkSZcZ6aiF4W5qagIsXqZA9JoasnAED6GdupG4fTU3kadi/n+of4+KoWb27xbEj3e2A42666xZu+/4BuKRUpC2cuONEWCQdjzWZsOa+iEVFxjFGrddGYEZEABMnkjs2JMT56+8uLS0klrW15DqWxDIxker2PO0E74nodPRZ+PZbIC+PLjJSUuRxsdSWrkoZPCGpR05ibg4WSSfCIulcLGXCKhRKzJt3wOQ4rupqylzcvZvmZgI0yePGG4Fbb5VHVx+djtzIWi25lDMyKGaZnk6uWk7y6YxeD5SV0UXSsWPkZUhNlV+NbVvMlTJ4SlKPO3qzOppjxzQICMjHk0/2QXKyfcLOImkGFknno9VqcPDg51iz5rlOf5NcsuZOEtKIo23bKEmGHkMt4yZPphIMT0cUaRJFZSVZQqmplA2bmUnJJ542sskd6PV0QXHkCHXNqaoCEhJoSos34knWmyetxR7aXmwoFAosXboUM2fafrFhixbw9S3jUKKikjFy5JQuk3pWrJiFn35a2y5xwM8PyM4GXnsNeOklyiAVReDoUUr8efNNOrHqLc+ldSuCQPWgffsCsbEk9ps3U5LPrl30e0MDvS9fQ6+ni4fdu4Fly+h4+PnRRYS3CiRguXTE1VhKPvJ0OmYQ6/V6zJ49GxqN9clH9iAzrz8jBzr2ijSV1COKenz44X0mrUpBIMsrM5OKyLdtoykPZ88CS5aQ+3XyZBJUT7bMQkJoa2oicdi3j0Q+PZ26+KSnk2tWbrE3W5Fc0cePk1tVmu2ZmekbbmhP60/qyt6sjsTUxYZOp0NBQYHdbldrYHerHbC71TqsbW9njbunqoqSfHbvpppFgJJ8br2VknwCAx2/fkej15MrVqslcYiPJ4uzXz/K4gwM9K5EHymp6fhx+r5cuUIu5/h436sttbU/KdMZU65ipVKJ4uJim0XSq2OS77//Pt5++22Ul5dj8ODBWLJkCUaNGmXVY1kk3UdX7e0ef3wtQkOju8z8a2ggi2zbNjrpAiQuEycCt9zimYN2TVFfTxZVYyNZkz17An36UNZeSAhleMpRMEWR/keFhcDJk0B+Pg26jooi97MvWI7mcFSDcV+GYpKzIYo6KJVKfPzxx06PScpKJD/77DM89NBD+OijjzB69Gi8++67+Pzzz5GXl4fY2NguH88i6V60Wo3J9naCIAAQDEk9U6YsRM+eIywKZmsr8MMPVNR/4QLdp1JRF5/bbpPP2C7JupQEPzycxLJnTyAtjTI9VSrPF8zmZnKp/vILcPo0cP48uVljYsi16unrZ+QDZbcW4MknMzi7tSOjR4/GyJEj8c9//hMABW5TUlLw1FNPYe7cuV0+nkXSM+jYhAAQYepjaE2qvF5PiT2bN1OtHUCuvKws4PbbKWtSLjQ1kVhWVVGsNTLS2G2mVy+ymAMCPMcaa2oiK7GwkOLFZ8/S7wEB1AhAzqUcjOfCdZJmaG5uRlBQENatW4d77rnHcP+0adNw9epVbNy4scvnYJH0HCTXU3X1JXz44X1m97M2PV0Uqcfn5s1kyQBkvQwfDvzqV2SZyYn6ehomXFtLiT0hIVRgn5hIZSU9epCFqVK5TjRbW0kYr16lvqolJbRVV9MaoqJI2D1FxBnvhKeAmKGyshI6nQ5xcXHt7o+Li8OZM2dMPqapqQlNTU2G36urq526RsZ6oqKSERWVDK1WY7FpsrWTBQSBut1cfz19iTZvNtbhHTpEhf133EFJMnJw/QUFGS2xxkayLk+fpouzgAASo4QESviJjye3pkpFFqifH1nT9r5PUSRXaUsLbVev0tiw8nJqF1dZSSLu50fJU+npnp1lzDDdQTYiaQ8LFizA66+/7u5lMBboWC7SEYVCCZUqGLm5e6xu55WeDjz1FMXFtmyh2OWpU7T17k1iOXiwPMQSIFGUXJh6Pbk0q6tJtESRknzUahLOHj0oeSkoiMZ5hYeTm9bf37RwiqLRQmxoIEGsqSELVqul5KKaGoo5iiIlGYWHA0lJvpehyvgmXu1uNWVJpqSksLvVA5Hcr0VFh7Bu3VxDqnxW1oPYv39Vt9p5VVQAX39NvUGlHrHJyeSGHTlS3id7KZu07SZ9o/38aFMoaFOp2tdk6vVkMTY1Ga3H1lbalEraV7JoAwO9v56TkQfsbjWDSqXC8OHDsWvXLoNI6vV67Nq1C08++aTJx6jVaqh5JIMskNyvmZnjMHr0/SbrK6VOPQEBocjIyLY6jT4mBnjoIeCuu4Dt24E9e8ht+PHHwJdfUjbsTTfJ02UoCO1dsxKiSNaf5DKVBLGx0fg4gMQzKIhE0d+/s5AyjK8jq6/Dc889h2nTpmHEiBEYNWoU3n33XdTV1WHGjBnuXhrjQCTBzM3dY1Onnq6IiAB++1uyIHftAnbsICvz//4P2LCBGhOMH+8dWZmCYHTDMu7BE6Z+MN1HViJ53333oaKiAn/+859RXl6OIUOGYOvWrZ2SeRjvwNIkdGnye3LyIJOTRSwRHExWZU4ONSbYupXib+vWUcLP+PHUmCAiwsFviJEl9oidp0z9YLqPbGKSjoBLQORHV516uposYg1SY4Kvv6bRTYCx2frkyfKqtWQcS1diZ0pA5T5pw9NxdUzS5oqmadOmYd++fXYvjmFsYcyYmXj77WI8/vjaLieLrFw5u91UEWvx86OY5BtvAH/4A138tLaSlTlvHvDee9RezXcuJxmg89SJjp+xffuW4YUX0rBo0QS88EIa9u1bBsCzpn4w3cdmd2tVVRUmTZqEtLQ0zJgxA9OmTUNSUpIz1sYwAChGOWrUFDQ2VlucLKLX61BQcMCqHrCmUCiAoUNpy8+n8pFjx6ijz9GjVFoyeTIwbJi8M2IZ6+hK7EwJ6IABOXZN/eD4pedis0hu2LABFRUVWLVqFVauXIn58+dj0qRJmDlzJu6++274yzFFkJEFbUf8mJosIgiCoS9sd+NAffoATz9N7tetW4EDB8jN88EHVMA/aRL1ifWGJB9vxV7hkR6nVoeYFTtLApqZOa5d7W9XMxt9OX6p11PZUl0dNahoaKBb6ee2W2OjsZa3f//uuVttodsxySNHjmD58uX497//jZCQEDz44IN44okn0KdPH0et0WFwTNK76KoHrCAo8Nhjn9pULmKOqioa07V7NxXaA1Tgf/PNJJhW9NdnXIi9wtPxcdnZU3HgwH87jbiyJu5ozdQPb4pf6vX03ZCaXUhNKaTb2loSQ+lWEkZ7FGjAADoH24vL6iQvXLiAHTt2YMeOHVAqlbj99ttx4sQJXH/99Vi0aBGeffbZ7jw9w1ikrWVpqgdsx3KRAQNy7HZphYcDv/41NU3fv59mW5aVURnJzp3AoEE0rqt/f+5d6m7MxRIHDMix+H839bgDB/6LefMOoLm5rp3YdewUZcpalEqZLGHJIvUUkZT69bbdqqrab5Io2mtyqVTGel+peYV0KzX2l36uqqL2k67CZpFsaWnBV199heXLl2P79u0YNGgQnnnmGTzwwAMGRV6/fj0efvhhFknG6VjTA5aaEDyKtuO47HVpqdVUIjJuHPVS3baNrmiPH6ctPp7+fuONVGrCuB57hcfc45qb65CZOa7T/m0v0uydEWlP/NKRtLTQiDOt1nir1RqHg1+9ShafLQQHU0vEtltISPstOJhEMCSEbm2J0hUWUn6Aq7BZJBMSEqDX6/G73/0OP/30E4YMGdJpn/HjxyOCi8wYF9JVD1hyw4rXfrbOsrCEIJDV2L8/9VDds4fa3pWXA59+CnzxBTBqFAlmr17y6RPrDdgrPPY8zhpr0RLWWKTdobXV2INX2i5fptvKSrIArUGlorrhyEjyqkRE0G14OPUKlm5DQryvY5PNMclVq1ZhypQpCAgIcNaanAbHJL0fc4OdTXH//YsxcuQUh52QGhspwWfPHuDcOeP9KSnAmDE045KtS9fQNl7dNpborMd1F2vil+ZoaQEuXQIuXqRb6WdJEPWWvwZQqagxflRU5y0ykrbAQM+50ON5kk6ERdJ3cORgZ1sRRaCggMTy4EFjU3V/f5pvedNNwHXXcezSGbTNaAVgVngsZb52R7CchV5PLtDycuDCBRJB6fbyZcuxQH9/6l8cHW28lbYePcj68xQBtAYWSSfCIulbtD3ZnTy5zeI4LmdlE9bVkXX5zTfUVF0iMpI6+mRn0yBlpvtYm9HqySUXra1kCZaV0XbhAt2Wl1PDenNIo9Ti4ijTWrqNiSFXqDddkLFIOhEWSdcgTZxobaUrXMndI33SFAq6clUouj8g2Ba0Wg0OHvwca9Y81+lvjz++1u4mBNYgikBREfD999QCr77e+LfUVHLFjh5N4snYjrWlFJ5SctHaSlbg+fMkgtLtxYv03TGFUknCFx/ffouLo3ignKzB7sCjshiPR6+ntPDGRuMmjWPS643zCxWK9gKoUBhFUxSNP+t0tI8gkGgGBlKcREr5dtSXPyoqGSNHTsFnn73gtCYE5hAEyshLTwfuv586+Bw4QBdbpaW0ffYZNTEYNYrcsiyY1mNtRqurSy70eqNlqNGQGJ4/T5ahOTEMCKB+wQkJ5GWQfo6J8b6kGDnAh5yxiF5PVk9NDbkO28bX1GqKZyQlkUsnMNCY2h0QQEKnUrW3HEWRnkOnI2FtaKDiYqnTRnU1xVhqaqgeqqWFHhsQYEwfV6nsfz8dswmN8Urbauq6g78/CeGoUfQ+Dx4k6zI/37h98gkJ6vDh1CYvPt4pS/EarMlM1Wo1qKmpcErJhShSuYRGYxRDjYbcpebcpAEB9N1JTDTeJiZSwoyvWIVygEWSaUdzs7FjhjSxPjCQ3Dmpqcbgf1QUCWNAgNFl2p24h2RVtrbS1tREYlleTgkLUsaelK0XGEivHxpq++t21YRAr9fh4MHPDZmvzuyrGRoKTJhAm1YLHDpEollQAJw9S9vatWRJDBpEW9++bFF0pKtSirZxSEAwCKU9JRe1tUaLsK11aK6e0N/fKITJyXSblMRiKBc4JmkH3hSTbG0lQayqIrepINCJW3LzxMdTHCQiwn1T61tbjeItJTNoNCSgDQ20Jqluy9b1mYpRSUhtyfbvX+XyJI8rV8gle+QIcOZMe9dcQACQmWms04yP962Tra2Zqab+x9SycA0yMrLMCmRDgzFe2Ha7etX0uhQKig9KQijdxsZ6V+KMu+GYJONURJHcmleu0BWxKJILs2dPqudLTqYvekiIbV0wnImfH21BQbS2gQONlmZxMQlmcTF9eRQKiuVFRFg3qcNSEwJR1OP771e2+33FilkICAg19IN1lpUZGWm0MOvrgVOnqKPPiRN0UXPsGG3Svn37Av360ZaQ4L2i2TEzdcqUhejZc4Th+Jsq7jcVhxRFPcLCYhAVlYzaWmMWadtNqzW/jh492luFycl03D3lO8M4DrYk7UBulmRrq7HnYnMzWSIxMXQ1lpJCVqM9VpgnIIokmFVV5KIsKaEs0tpasnyjoqzL/LOU+WoKd1mZej0l+Zw6RW3xfvnFGCeWCA6meGZGBt2mpZF3QO50ZfWbO/5arQbPP98bQBqAvgD6AchEevpDqKhQo6bG/GtGRBhdpZIYJiaSu59xD1wC4kR8SSSbmshavHqVhCQ0lE6W6ekkjLGxlHjjTRaHKJLLWKs1JsCUlZHbLDycBFOtNv94Sydha3BHKUFTE8Utf/kFyMujn1taOu/Xowf9/9taP3Fx8rowys3dg0WLJpj9uyBE4Pnnc9HUFI+KCmP3mUuXgIoKPUTRvM8zKooswbYJNImJ3CHJE2F3K2M39fXGzFCFgk6Mo0eTMCYnW++ClCuCQFf4SUl0whs5ktxoZ86QiJSW0j7R0SSaHS8QTCV/ZGU9aBiV1BXdHfpsD2o1TUSQpiK0tlJLvLNnybIuLjYmPF2+TDFOCaWSPApSEbrUjaVHD9o8pRWZKNKFDnAdgIkA4gAkA0i5dpsKoCdEMQrvvGPuWRRQqfSIiKhHQoICaWlBiIsjIYyPJ+8Kw5iCLUk78BRLUhQpo+7yZaN7MTaWau0kYQwO9owTnTtpaaELB8naKimhC4rwcBKFjnGkjskf1vaDFQQBliaNuGv6fEMDXSCUlLTNxtSjqclyNonU1DoiglzWwcEUq1YorqK5uRwxMTGIju4BtZqOoRQ7lmpjpc+dVD8rZS+3tNDW3GwcpisN1JXmDNbV0f9MyrTu6FI2R1CQDrGxSsTEGDvOSBcBERHe8V1w1+fIU2BLkrGIKJIgVlbSiT4wkE4A2dkkjBwv6Yy/P7nTRo+mEoqyMiA3l7aiIvp7bKzRtdYx+SMqKhmjRk1BY2O1WSvTVL1l2ySfkye3ua0VWmCgMakHoOSX5ctnA0gE0A9ZWa8hLOxGQ0PsykoSqeZmo7uyPRHXNtcSEKBHZKQCwcGNUKuvQqcrRl7eKohiCQThHH7/++cxceJDLl+XK/HklnreCluSduBqS1IUjUX29fV0Mk9KIouxVy8Sxu4U2Psizc0Urz1zhsTy/HmybqKjKVvUXMq+KSvTXL2lEQGCII3runaPoMBjj31qyJJ1Fda2ZZMG7VZV0W11NVBRUYXt21cCiAIQCiAMQChiYwdDFP0NFqLUTUna2nZg8vOjixJ/f/rMBgQAtbUlOHfuOwBXAFy9dnsZwGUMGjQCP/+8EkA5BKHJpHXuac3InYWntNRzN2xJMgDINVVVRVf1zc3k5kpLo1T/nj0p5ianpAtPQ3JNR0cDQ4ZQ7E7KFq2spESn6OjOFx+mrMyuhj6ThdnhHlGPDz+8z+XWgLVt2dRqo5tSIjf3CLZvf7rTc86YscfkUGJr3IJ04k8HYNqN/fPPmww/iyI6dUPq7jxHOeHqlnoMwadZD0KnM04Eb2mhE3XfvrSlpdEJy5sTb9yBQkHxtoEDyTK/eJHEMjeXhFMaMxQSYvl5uhr6bA5XtMFri70DiW19rLVuQVMnfkv4sih053/H2A/3gXAzLS10Yj5zhpJKGhvphD1lCvDYY8CDD9KEiMREFkhnImXG9uwJ3HILuXJuu42Sny5dov9NZaXlAbZjxszE228X4/HH116LUbZ9fgGCYPofKJ34XYEk5goFrcWWtmzWPlar1bRpAWe8ENBqNZ2eUzrxW4uzREGr1SA3d4/JNXoK3fnfMfbDlqQbaGgga7Gqik7OkZHUyDojg07S3jb/TW74+5Or9cYbqbl4URFZlwUF5I4NDqa/m0qQMpfkM23axxgwIMdklqypRtzOzF5s27vW1lheV4/VajX46ae1VrsFOzecN2YId0yOcpYoyCkZpjv/O8Y+OHHHDmxN3JEyUi9fpqxBqRNM376UkZqaKr/p4L5GYyNQUWHMiq2spP9rjx7mSwvMJZXs27esk4BKJ2V3nbAdIcztm4i3p6sEk7bHCoDJ5ChniAInw8gPTtwxw1/+8hds3rwZx44dg0qlwlVzXYY9BKkV3JUr5FINDKSi5T59yFpMTuYCZjkREECdiqQmBfn55CIvKSF3bEgICWbb/6m5pBJz1oA5N6Wz45VdCbO1CTiWBLIrC9BUQpS5vzkSToZhukI2Itnc3IwpU6YgKysLy5Ytc/dyOtG2cXh1NVkWYWHAddeRxZmSIr82YExn/PzIPT5yJDBgAE0ikTr6aDT0OYiKslxGApg+8Vt7wrbW6rNH3DoKc3cTcH73u79jxIh7PVZwOBmG6QrZnLJff/11AMCKFSvcu5A2tLRQXFGyFtVqY0mBZC2GhXF80RsRBJpKIvXCzcoy9ostKiJ3rPR5sKbrkbUDga0Vre6IW9tEImstW3Ni48kCCXQ9h5JhZCOS9tDU1ISmpibD79XV1Q577qtXyaUqWYtSG7i4OMtNtBnvw9+/vXVZWUmWZX4+ZS6XlZE7NjLSdLKPtQOBrXXH2uK2tWRJ2eKKlLPYcDIMYwmvFskFCxYYLFBHkpFBJ7yUFOp8Q/0sHf4yjMyQrMvUVCrZGT3aOLorP5+aret0dGEVGUkJXJ1jeSIAAY8/vrbTQGBrRMtSdqmp5utdiZstrkg5i40vNSVgbMOtIjl37lz89a9/tbhPbm4uMjMz7Xr+l19+Gc89Z5wPWF1djZSUFLueqy3Dh3f7KRgvx8+PmkEMGEBZzDfeSFl5JSU0oaO0lOKXlZWWBwK3pav4maXsUkEQDKUnHd2v5sTNHuuQxYbxNtwqks8//zymT59ucZ/09HS7n1+tVkPNvk/GzUglP5GRJJq1tSSURUXAzz9bnzhiSbQsZZeaar5uqb1b24QfOVuHDOMI3CqSMTExiImJcecSGMZlCAKViAQEULnIoEHAhAnJEISlWLhQKqZXYvLkj9HUlIz6enLfWhItgIYR19RUmM0ujYhI7NR83Vxs0VzCD4sj46vIJiZZWloKrVaL0tJS6HQ6HDt2DACQkZGBkK4aazKMhyG1wQsMBN58cyamT8/B6dMFUCoz0NycjKIiKi85dGgZtm41LVqdE36EdpNGpOxSer2urVV76jRtaULQdl8APj0TkZEPshHJP//5z1i5cqXh96FDhwIA9uzZg3HjxrlpVQzTfQQB6NMnGX36JEMUaUxVfT1w6JAGb77ZXrRWrJiNlJQchITAZMKPICghip3jh9bEFm0trLelO1BnQafRYZ7eBo5hZCOSK1as8KgaSYZxBm1dsv7+ppJ6dNBoCtDYKJr4m4jHH/8UYWExneKH1sQWbSmst8XqNJXBKxm8rp6CwjC2woULDOOh9OnTB4oOtUVKpRKzZmXg17/uPD1DEJRISspCZuY4k4ITFZVs9m/S362dMtFVE4Ku9rXmcQzjCcjGkmQYXyM5ORlLly7F7NmzodPpoFQq8fHHH2P06GSMHAm8//5SPPUU/U2hUOJ//ocSfvLyqHY3PNy6bj9tsTab1Rar09S+beE2cIwnw1NAGMbD0Wg0KCgoQEZGBpKTkzv9LT+/ACkpJGilpcC5c1ReotVSfDMwkEpQbBXMrrA0zcTSvh3HYVl6HMN0xNVTQFgkGcbLaG6mmaUajbHjT0WFcwTTljFWlsZhMYy18KgshmG6hUpFW3g4dftpbATOnweKi6mJQUUF/R4cTIIZFGT/a9nSYcfSOCyG8VRYJE2g0+nQ0tLi7mXIEn9/fyiVSncvg7mGvz9tmZnUc/jGG8nCLCwECgqoFrOlhQZHS/1kGYYxwiLZBlEUUV5e7vEDnT2diIgIxMfHX4s9MZ6C1E9WmnF6003GXrL5+WRpKhQkll3Nw2QYX4FFsg2SQMbGxiIoKIhP8jYiiiLq6+tx6dIlAEBCQoKbV8SYw9+f3LGDBpGVWV1NYllQQFZmRQXFL2NiuueOZRi5wyJ5DZ1OZxDIHj16uHs5siXw2sDES5cuITY2ll2vMkClouHQUj9ZaR5mXh6N92ppob9FRQH872R8DRbJa0gxyCC+bO420jFsaWlhkZQRUj9ZaU7qqFHkjpUEMz+fOgHFxFDSD8P4AiySHWAXa/fhYyh/FIrO8zBzc2mTrMvoaI5dMt4PiyTDMBZRqYC4OCA2FhgxgqzL06fJurx0ibr7xMYCPLqV8UZYJLugoYGKs12FSkUuL0eTl5eHsWPHIj8/H6GhoQCADRs24IUXXkBRURGeeuopTJ48GXPnzsWRI0c69QxlGEEgQezfH+jTB7h8mcTy5EkSToWCxDI01LGdfRjGnbBIWqChAdi4EbhyxXWvGRkJ3H2344Xy5ZdfxlNPPWUQSACYPXs2ZsyYgT/84Q8IDQ1FaGgoXn31VaxevRpTp0517AIYr0KlAhISyMIcOZKsypMnqbvP+fPGRCC+1mLkDoukBZqbSSADAylhwdk0NtLrNTc7ViRLS0uxadMmLFmyxHBfbW0tLl26hJycHCQmJhrunz59Ot577z0WScYqFAogLIyEcuBA6ht76hRtUqP1uDhuUsDIF77Os4KAAMrmc/ZmrxDX1dXhoYceQkhICBISEvC3v/0N48aNwzPPPAMAWLt2LQYPHoykpCQAwN69ew0W5YQJEyAIAvbu3QsAuPPOO3Ho0CGcPXu2u4eN8TECAsgNe8cdwMyZwC230Oe6sJBqMOvr3b1ChrEdFkkv4MUXX8Q333yDjRs3Yvv27di7dy+OHDli+Pu3336LESNGGH7Pzs5GXl4eAOCLL77AhQsXkJ2dDQBITU1FXFwcvv32W9e+CcZr8PMD4uOBiROBRx4Bfv1rIDUVKCsDzpwBrl4FfGesAiN32N0qc2pra7Fs2TL897//xcSJEwEAK1eubDdSqaSkpJ1IqlQqxMbGAgCioqIQHx/f7jkTExNRUlLigtUz3owgkCt29GhqUlBYCBw/TnWXZWUUs4yJ4bgl49mwSMqcs2fPorm5GaNHjzbcFxUVhX79+hl+b2hoQIANvtzAwEDUs2+McSCBgZQV27cvJfYcP06JPmfOUHu8uDiyQBnG0+CPpQ8QHR2NKzak6Gq1WsTExDhxRYyv4u8P9OxJXX1uuAE4ccJoXQYGUsasK5LkGMZa2NEhc3r37g1/f3/8+OOPhvuuXLmCX375xfD70KFDcfr0aauer7GxEWfPnsXQoUMdvlaGkVAqyXqcOJGSfO65h34vLqYm63V17l4hwxBsSVpBY6Pnvk5ISAhmzpyJF198ET169EBsbCzmzZvXrhlATk4OHnnkEeh0ui57qf7www9Qq9XIysqyfTEMYyOCQLMss7OBIUOobOToUYpftrZSAlB4ODcnYNwHi6QFVCoq7r9yhRoLuAJ7Bt++/fbbqK2txZ133onQ0FA8//zzqKqqMvz9tttug5+fH3bu3ImcnByLz/Xpp5/i97//PTd6Z1xOUBAwdChw/fUkkkePkmieP0+dfKKjWSwZ18MiaYHAQOp+4+lt6UJCQrBq1SqsWrXKcN/mzZsNP/v5+eGVV17B4sWLDSIZEREBsUMefmVlJdatW4dDhw7Z/wYYppuo1TQYuk8fEsgjRyjJ59QpEsrYWM6IZVwHi2QXBAY6p5eqq5k9ezauXr2Kmpqadq3p2lJcXIwPPvgAvXr1cvHqGKYzfn5AWpoxyef4cdpOnyYXbEICZ8Qyzoc/Yj6Cn58f5s2bZ3GfESNGtKunZBhPQKEgQYyPp/Z3J06QK1bKiE1M5AkkjPOQhUgWFxfjjTfewO7du1FeXo7ExEQ8+OCDmDdvHlTcFNIkUps5hvEWBIEaEIwbBwwbRhblkSMUv/T3J7HkUDrjaGQhkmfOnIFer8fHH3+MjIwMnDx5Eo8++ijq6urwzjvvuHt5DMO4mLAwcsEOHkwNCQ4fJrEURRJLMxEFhrEZWYjk5MmTMXnyZMPv6enpyMvLw4cffsgiyTA+TGAgZcT270/1lYcOAfn5gEZD7tmICM6IZbqHLETSFFVVVYiKirK4T1NTE5qamgy/V1dXO3tZDMO4AZWKSkf69qUB0AcPkoVZVsblI0z3kGUidUFBAZYsWYLZs2db3G/BggUIDw83bCkpKS5aIcMw7sDPD+jdG/jtb4EZM4CbbqImHadOAeXlgF7v7hUycsOtIjl37lwIgmBxO3PmTLvHnD9/HpMnT8aUKVPw6KOPWnz+l19+GVVVVYbt3Llzznw7DMN4CAoFlY/cdReN65o4kQTy9GmqvdTp3L1CRi641d36/PPPY/r06Rb3SU9PN/xcVlaG8ePHIzs7G0uXLu3y+dVqNdScG84wPosgtC8f+flnSvI5cwYICaEkH39/d6+S8WTcKpIxMTFWT5s4f/48xo8fj+HDh2P58uXtepM6E53OtS4ahYKaPzuSvLw8jB07Fvn5+WYbCbSlsrIS119/PY4cOdJuLiXDyBVBoLjkhAlUPnLiBImlVGuZnGx7O0jGN5BF4s758+cxbtw4pKWl4Z133kFFRYXhbx0HBjsSnQ44d871belSUhwrlC+//DKeeuopqwQSoNFaDz30EObPn49ly5Y5biEM4wFERAA330xZsadOUZLP2bMUz0xK4lpLpj2yEMkdO3agoKAABQUFnSybjv1HHYleTwKpVLqm/VVrK72eXu84kSwtLcWmTZuwZMkSmx43Y8YMDB8+HG+//XaXWcQMI0dCQoDRo4FBg4DcXCofKSqivyUnA8HB7l0f4xnIIrt1+vTpEEXR5OYK/PwobuHszR4hHjduHJ588kk8+eSTCA8PR3R0NF599VXDsVm7di0GDx6MpKQkAEBdXR3CwsKwbt26ds+zYcMGBAcHo6amBgDQv39/JCYmYv369d07eAzj4QQGkgt2+nTggQeojKSsjISzzTAdxkeRhUgyllm5ciX8/Pzw008/4R//+AcWL16Mf//73wCAb7/9tl0/1uDgYNx///1Yvnx5u+dYvnw57r333nYu2VGjRuHbb791zZtgGDejUgEDBwJTpwIPPUQWZmUlZcRqtdTNh/E9ZOFuZSyTkpKCv//97xAEAf369cOJEyfw97//HY8++ihKSko6NS1/5JFHkJ2djQsXLiAhIQGXLl3Cli1bsHPnznb7JSYm4ujRo658Kwzjdvz8gH79aFRXcTG5YXNzybqMi+PGBL4GW5JewA033AChzbc2KysL+fn50Ol0aGhoQEBAQLv9R40ahf79+2PlypUAgP/+979IS0vDmDFj2u0XGBiI+vp6578BhvFAFAogPR24917g4Ycp2aexkWZbXrzIjQl8BRZJLyc6OhpXrlzpdP8jjzyCFStWACBX64wZM9oJLQBotVqrS3QYxltRKCjjXGpMMGkSNybwJVgkvYAff/yx3e8//PAD+vTpA6VSiaFDh+L06dOdHvPggw+ipKQE7733Hk6fPo1p06Z12ufkyZMYOnSo09bNMHJCakxw223Ao48Ct99OccwzZ4DSUspOZ7wPFkkraG0FWlqcv9n7JSstLcVzzz2HvLw8fPrpp1iyZAmefvppAEBOTg4OHDgAXYfL3cjISPzmN7/Biy++iFtvvbVTaU19fT0OHz6MW2+91b5FMYwXIzUmePRRsjDDwkgsi4pcW1fNOB9O3LGAQkFXis3NrnOpqFT0urbw0EMPoaGhAaNGjYJSqcTTTz+NWbNmAQBuu+02+Pn5YefOncjJyWn3uJkzZ+KTTz7Bww8/3Ok5N27ciNTUVNx88812vxeG8XakxgRtu/icPctDoL0JFkkLKJUUi/D0tnT+/v5499138eGHH3b6m5+fH1555RUsXry4k0ieP38ePXr0wN13393pcf/4xz/w5z//2baFMIyPEhxMQ6CHDKFM2IMHyaoUReriw0Og5QuLZBcolY7vpepqZs+ejatXr6KmpgahoaGor6/HhQsXsHDhQsyePRuqDk0rKysr8Zvf/Aa/+93v3LRihpEnAQHU7m7AAOoLe+gQ3Z47R/HMyEh3r5CxFY5J+gB+fn6YN2+eoVHAokWLkJmZifj4eLz88sud9o+OjsYf//jHTtmuDMNYh78/0L8/8OCDNNdy+HDq3nPyJFBRwY0J5ARbkjJn7969Nj/mtddew2uvvebwtTAM0x6lEsjIoEHQpaXAsWMklCdPAj160AgvFw00YuyERZJhGMbJCAINgU5Npdjlzz8DR49SrWVoKM+19GRYJBmGYVyEIFBru1tuAUaNIovy0CEgPx9QqyluyRmxngWLJMMwjBsIDwduvJESfc6cIbEsLqZs+vh4Ki9h3A+LJMMwjBsJCqI6y4EDqcbyyBEgLw/QaICYGNo4buk+WCQZhmE8AH9/IDOTJpCcO0fNCY4fp7hlWBhZlx2qtRgXwCLJMAzjQQgCJfikpgJZWSSSUicfpZLiltycwHWwSDIMw3goUVHATTdRnWV+PmXEnj1LscvYWHbFugI+vF7A9OnTIQgCBEGAv78/4uLicMstt+A///kP9Db01FuxYgUiOFuAYTyOwEBg0CBjc4Jx4yjB59Qpan/X0ODuFXovLJJewuTJk3HhwgUUFxfj66+/xvjx4/H000/jjjvuQCvP8GEYr0CpBHr1Au68E3jsMRoIHRdHjQpOnQIqK7mbj6NhkfQS1Go14uPjkZSUhGHDhuGVV17Bxo0b8fXXXxuGKy9evBgDBw5EcHAwUlJS8MQTT6C2thYAde6ZMWMGqqqqDFap1JVn1apVGDFiBEJDQxEfH48HHngAly5dctM7ZRgGoD6wWVk0rmvGDPq5qYlqL4uKgLo6d6/QO2CRdAIajQZ79uyBRqNx6zomTJiAwYMH48svvwQAKBQKvPfeezh16hRWrlyJ3bt3449//CMAIDs7G++++y7CwsJw4cIFXLhwAS+88AIAoKWlBW+88QaOHz+ODRs2oLi4GNOnT3fX22IYpg3+/kDfvsCvf03W5f/8D1mXZWVkXV64wAOhuwMn7jiYZcuWYdasWdDr9VAoFFi6dClmzpzptvVkZmbi559/BgA888wzhvt79uyJN998E4899hg++OADqFQqhIeHQxAExMfHt3uOtvMm09PT8d5772HkyJGora1FSEiIS94HwzCWEQTqB5udDYwcSS7Y3FwSyl9+IVdtTAxZoDy7wHrYknQgGo3GIJAAoNfrMXv2bLdalKIoGqZ57Ny5ExMnTkRSUhJCQ0MxdepUXL58GfX19Raf4/Dhw7jzzjuRmpqK0NBQjB07FgBQWlrq9PUzDGM7/v7UVP2OO4AnngAeeICaFdTVkWgWFADV1Ry/tAYWSQeSn5/fKZtUp9OhoKDATSsCcnNz0atXLxQXF+OOO+7AoEGD8MUXX+Dw4cN4//33AQDNzc1mH19XV4ecnByEhYVh9erVOHjwINavX9/l4xiG8QxCQ2kY9O9+B8yeTe7YXr2Ay5dJMAsLWTAtwe5WB9KnTx8oFIp2QqlUKpGRkeGW9ezevRsnTpzAs88+i8OHD0Ov1+Nvf/sbFNcKq9auXdtuf5VKBZ1O1+6+M2fO4PLly1i4cCFSUlIAAIcOHXLNG2AYxmEIAtVWxsYCo0dTrLKoiPrGnjtHm1pNtZkREVx/KSGbw3DXXXchNTUVAQEBSEhIwNSpU1FWVubuZbUjOTkZS5cuhVKpBEAC+fHHHyM5Odnpr93U1ITy8nKcP38eR44cwVtvvYW7774bd9xxBx566CFkZGSgpaUFS5YsQWFhIVatWoWPPvqo3XP07NkTtbW12LVrFyorK1FfX4/U1FSoVCrD47766iu88cYbTn8/DMM4D4UCSEqiRgUPPwzMmkWJPxkZ5JLNzSXxvHiRMmZ9GdmI5Pjx47F27Vrk5eXhiy++wNmzZ3Hvvfe6e1mdmDlzJoqLi7Fnzx4UFxe7LGln69atSEhIQM+ePTF58mTs2bMH7733HjZu3AilUonBgwdj8eLF+Otf/4oBAwZg9erVWLBgQbvnyM7OxmOPPYb77rsPMTExWLRoEWJiYrBixQp8/vnnuP7667Fw4UK88847LnlPDMM4H0kws7OBadOAxx8n1+zw4fT3oiJjWYlWS00MfAlBFOXpif7qq69wzz33oKmpCf5WTiutrq5GeHg4qqqqEBYW1u5vjY2NKCoqQq9evRAQEOCMJfsMfCwZxjuor6dpJOfOUTu8ixcpfgnQ9JLwcNquOc9cQmEhJSVNm2b/c1jSgo7IMiap1WqxevVqZGdnWxTIpqYmNLXxFVRL/12GYRimS4KCqAazb19gwgSyJMvKKJ5ZWAhUVJBw6vWUURsaCoSEAMHB3hPTlJVIvvTSS/jnP/+J+vp63HDDDdi0aZPF/RcsWIDXX3/dRatjGIbxXqQ6zB49qJxErweqqkgkKypIPDUayprVaOjvSiX1nQ0IIOEMCAD8ZKU6bna3zp07F3/9618t7pObm4vMzEwAQGVlJbRaLUpKSvD6668jPDwcmzZtMtQBdsSUJZmSksLuVifDx5JhfJPmZrI2r16l20uXgPJyEtOGBqCxEZAS6AWBrE9/f5qT6edn3BSK9htAJSqiSK7fvn19xN36/PPPd9neLD093fBzdHQ0oqOj0bdvX1x33XVISUnBDz/8gKysLJOPVavVUKvVjlwywzAMYwaVioZDt23aJYqUIVtVBdTUUPZsXR0JaXU13VdfTwLa0EAiKrXRk5KERNHYJUipJIvUVbhVJGNiYhATE2PXY6VaxCYH5yfLNI/Jo+BjyDCMhCCQqAUEUE9ZU+j1JIxNTUBLC22trSSOej1tbS3LLow/hyIL7/CPP/6IgwcP4qabbkJkZCTOnj2LV199Fb179zZrRdqKlABUX1+PwMBAhzynryK1ubM265hhGN9GoSArVKVy90o6IwuRDAoKwpdffon58+ejrq4OCQkJmDx5Mv70pz85zJ2qVCoRERFhGAEVFBRkNtbJmEYURdTX1+PSpUuIiIgwNFVgGIaRK7IQyYEDB2L37t1Ofx1p+gXPSuweERERnSaJMAzDyBFZiKSrEAQBCQkJiI2NRUtLi7uXI0v8/f3ZgmQYxmtgkTSBUqnkEz3DMAwjn96tDMMwDONqWCQZhmEYxgwskgzDMAxjBp+KSUpF7tzonGEYxneRNMCaxic+JZI1NTUAgJSUFDevhGEYhnE3NTU1CA8Pt7iPbOdJ2oNer0dZWRlCQ0PtbhQgNUk/d+5cl41xfQ0+Nqbh42IePjam4eNiHkccG1EUUVNTg8TERCi6mOnlU5akQqFAcnKyQ54rLCyMP7xm4GNjGj4u5uFjYxo+Lubp7rHpyoKU4MQdhmEYhjEDiyTDMAzDmIFF0kbUajXmz5/PcypNwMfGNHxczMPHxjR8XMzj6mPjU4k7DMMwDGMLbEkyDMMwjBlYJBmGYRjGDCySDMMwDGMGFkmGYRiGMQOLpAnef/999OzZEwEBARg9ejR++ukni/t//vnnyMzMREBAAAYOHIgtW7a4aKWux5Zj869//Qs333wzIiMjERkZiUmTJnV5LOWKrZ8ZiTVr1kAQBNxzzz3OXaAbsfXYXL16FXPmzEFCQgLUajX69u3rld8pW4/Lu+++i379+iEwMBApKSl49tln0djY6KLVuoZ9+/bhzjvvRGJiIgRBwIYNG7p8zN69ezFs2DCo1WpkZGRgxYoVjl2UyLRjzZo1okqlEv/zn/+Ip06dEh999FExIiJCvHjxosn9v//+e1GpVIqLFi0ST58+Lf7pT38S/f39xRMnTrh45c7H1mPzwAMPiO+//7549OhRMTc3V5w+fboYHh4uajQaF6/cudh6XCSKiorEpKQk8eabbxbvvvtu1yzWxdh6bJqamsQRI0aIt99+u/jdd9+JRUVF4t69e8Vjx465eOXOxdbjsnr1alGtVourV68Wi4qKxG3btokJCQnis88+6+KVO5ctW7aI8+bNE7/88ksRgLh+/XqL+xcWFopBQUHic889J54+fVpcsmSJqFQqxa1btzpsTSySHRg1apQ4Z84cw+86nU5MTEwUFyxYYHL/3/72t+KvfvWrdveNHj1anD17tlPX6Q5sPTYdaW1tFUNDQ8WVK1c6a4luwZ7j0traKmZnZ4v//ve/xWnTpnmtSNp6bD788EMxPT1dbG5udtUS3YKtx2XOnDnihAkT2t333HPPiTfeeKNT1+lOrBHJP/7xj2L//v3b3XffffeJOTk5DlsHu1vb0NzcjMOHD2PSpEmG+xQKBSZNmoQDBw6YfMyBAwfa7Q8AOTk5ZveXK/Ycm47U19ejpaUFUVFRzlqmy7H3uPzv//4vYmNjMXPmTFcs0y3Yc2y++uorZGVlYc6cOYiLi8OAAQPw1ltvQafTuWrZTsee45KdnY3Dhw8bXLKFhYXYsmULbr/9dpes2VNxxfnXpxqcd0VlZSV0Oh3i4uLa3R8XF4czZ86YfEx5ebnJ/cvLy522Tndgz7HpyEsvvYTExMROH2o5Y89x+e6777Bs2TIcO3bMBSt0H/Ycm8LCQuzevRu///3vsWXLFhQUFOCJJ55AS0sL5s+f74plOx17jssDDzyAyspK3HTTTRBFEa2trXjsscfwyiuvuGLJHou58291dTUaGhoQGBjY7ddgS5JxCQsXLsSaNWuwfv16BAQEuHs5bqOmpgZTp07Fv/71L0RHR7t7OR6HXq9HbGwsli5diuHDh+O+++7DvHnz8NFHH7l7aW5l7969eOutt/DBBx/gyJEj+PLLL7F582a88cYb7l6a18OWZBuio6OhVCpx8eLFdvdfvHgR8fHxJh8THx9v0/5yxZ5jI/HOO+9g4cKF2LlzJwYNGuTMZbocW4/L2bNnUVxcjDvvvNNwn16vBwD4+fkhLy8PvXv3du6iXYQ9n5mEhAT4+/tDqVQa7rvuuutQXl6O5uZmqFQqp67ZFdhzXF599VVMnToVjzzyCABg4MCBqKurw6xZszBv3rwuZyJ6K+bOv2FhYQ6xIgG2JNuhUqkwfPhw7Nq1y3CfXq/Hrl27kJWVZfIxWVlZ7fYHgB07dpjdX67Yc2wAYNGiRXjjjTewdetWjBgxwhVLdSm2HpfMzEycOHECx44dM2x33XUXxo8fj2PHjiElJcWVy3cq9nxmbrzxRhQUFBguHADgl19+QUJCglcIJGDfcamvr+8khNKFhOjD7bddcv51WAqQl7BmzRpRrVaLK1asEE+fPi3OmjVLjIiIEMvLy0VRFMWpU6eKc+fONez//fffi35+fuI777wj5ubmivPnz/fqEhBbjs3ChQtFlUolrlu3Trxw4YJhq6mpcddbcAq2HpeOeHN2q63HprS0VAwNDRWffPJJMS8vT9y0aZMYGxsrvvnmm+56C07B1uMyf/58MTQ0VPz000/FwsJCcfv27WLv3r3F3/72t+56C06hpqZGPHr0qHj06FERgLh48WLx6NGjYklJiSiKojh37lxx6tSphv2lEpAXX3xRzM3NFd9//30uAXEFS5YsEVNTU0WVSiWOGjVK/OGHHwx/Gzt2rDht2rR2+69du1bs27evqFKpxP79+4ubN2928Ypdhy3HJi0tTQTQaZs/f77rF+5kbP3MtMWbRVIUbT82+/fvF0ePHi2q1WoxPT1d/Mtf/iK2tra6eNXOx5bj0tLSIr722mti7969xYCAADElJUV84oknxCtXrrh+4U5kz549Js8Z0rGYNm2aOHbs2E6PGTJkiKhSqcT09HRx+fLlDl0Tj8piGIZhGDNwTJJhGIZhzMAiyTAMwzBmYJFkGIZhGDOwSDIMwzCMGVgkGYZhGMYMLJIMwzAMYwYWSYZhGIYxA4skwzAMw5iBRZJhGIZhzMAiyTAMwzBmYJFkGB+goqIC8fHxeOuttwz37d+/HyqVqtMUBYZhjHDvVobxEbZs2YJ77rkH+/fvR79+/TBkyBDcfffdWLx4sbuXxjAeC4skw/gQc+bMwc6dOzFixAicOHECBw8ehFqtdveyGMZjYZFkGB+ioaEBAwYMwLlz53D48GEMHDjQ3UtiGI+GY5IM40OcPXsWZWVl0Ov1KC4udvdyGMbjYUuSYXyE5uZmjBo1CkOGDEG/fv3w7rvv4sSJE4iNjXX30hjGY2GRZBgf4cUXX8S6detw/PhxhISEYOzYsQgPD8emTZvcvTSG8VjY3cowPsDevXvx7rvvYtWqVQgLC4NCocCqVavw7bff4sMPP3T38hjGY2FLkmEYhmHMwJYkwzAMw5iBRZJhGIZhzMAiyTAMwzBmYJFkGIZhGDOwSDIMwzCMGVgkGYZhGMYMLJIMwzAMYwYWSYZhGIYxA4skwzAMw5iBRZJhGIZhzMAiyTAMwzBmYJFkGIZhGDP8f0YOUIOEcj0xAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "train_and_test_approximate_qep(qpytorch.mlls.PredictiveLogLikelihood)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.12" } }, "nbformat": 4, "nbformat_minor": 4 }