{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# QEP Regression with Uncertain Inputs\n",
    "\n",
    "## Introduction\n",
    "\n",
    "In this notebook, we're going to demonstrate one way of dealing with uncertainty in our training data. Let's say that we're collecting training data that models the following function.\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "y &= \\sin(2\\pi x) + \\epsilon \\\\\n",
    "  \\epsilon &\\sim \\mathcal{Q}(0, 0.2) \n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "However, now assume that we're a bit uncertain about our features. In particular, we're going to assume that every `x_i` value is not a point but a distribution instead. E.g.\n",
    "\n",
    "$$ x_i \\sim \\mathcal{Q}(\\mu_i, \\sigma_i). $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Using a distributional kernel to deal with uncertain inputs\n",
    "\n",
    "Rather than using a variational method (see the QEP Regression with Uncertian Inputs tutorial in the variational examples), if we explicitly know the type of uncertainty in our inputs we can pass that into our kernel.\n",
    "\n",
    "More specifically, assuming Gaussian inputs, we will compute the symmetrized KL divergence between the Q-exponential inputs.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import math\n",
    "import torch\n",
    "import tqdm\n",
    "import qpytorch\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "%matplotlib inline\n",
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Training data is 100 points in [0,1] inclusive regularly spaced\n",
    "train_x_mean = torch.linspace(0, 1, 20)\n",
    "# We'll assume the variance shrinks the closer we get to 1\n",
    "train_x_stdv = torch.linspace(0.03, 0.01, 20)\n",
    "\n",
    "# True function is sin(2*pi*x) with Gaussian noise\n",
    "train_y = torch.sin(train_x_mean * (2 * math.pi)) + torch.randn(train_x_mean.size()) * 0.2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "To effectively pass in the training distributional data, we will need to stack the mean and log variances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "train_x_distributional = torch.stack((train_x_mean, (train_x_stdv**2).log()), dim=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x124cb7b20>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqsAAAESCAYAAADaNpzRAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAJ0VJREFUeJzt3Xt01OWdx/HPb4AZBnLhOhkwiVzUYuUSnSgNpWtJaWlxaWE9J1RZQQ8SrbJnFV2EouKlBaVIPWVxUxEX4dBEuwWPKywVKZ5ugIUOMltbgV2BkHQhEKgSEjQhmWf/sJkmGCCTzG/ml8n7dc6cmMnv8o2PxA9Pnt/3sYwxRgAAAIADuRJdAAAAAHAphFUAAAA4FmEVAAAAjkVYBQAAgGMRVgEAAOBYhFUAAAA4FmEVAAAAjtU90QXEWjgc1vHjx5WamirLshJdDgAAAC5ijNG5c+c0ePBguVyXnztNurB6/PhxZWVlJboMAAAAXEFFRYUyMzMve0zShdXU1FRJn3/zaWlpCa4GAAAAF6uurlZWVlYkt11O0oXVpl/9p6WlEVYBAAAcrC1LNnnACgAAAI5FWAUAAIBjEVYBAADgWEm3ZhUAAHQujY2NunDhQqLLQAz16NFD3bp1i8m1CKtAkgsGg5o/f76WLVum3NzcRJcDABHGGFVWVuqTTz5JdCmwQZ8+feT3+zvc956wCiS5devWaceOHVq/fj1hFYCjNAVVn8+nXr16sZlPkjDG6Pz58zp16pQkadCgQR26HmEVSKDa2lpbrlteXq4zZ87IsiyVlJRIkoqLi1VQUCBjjPr376/s7OyY37d3794xvyaA5NTY2BgJqv379090OYgxr9crSTp16pR8Pl+HlgQQVoEESklJidu9qqqqNH78eFvvYYyx9foAkkfTGtVevXoluBLYpWlsL1y40KGwSjcAAACQMPzqP3nFamyZWQUSqKamxrZrh0KhVmdSS0tLlZOTY9t9AQCIJcIqkEB2rvFsWi/kcrkUDocjH71eL2tLAQCdBssAgCTl8/nk9/sVCARUVFSkQCAgv98vn8+X6NIAAM0MGTJEL774YqLLcCzCKpCkMjMzVVZWpj179ui+++7Tnj17VFZWpszMzESXBgAxFQwGlZ+fr2AwaOt9LMu67Oupp55q13V/97vfqbCwsEO1ff3rX4/U4fF4dNVVV2nKlCnauHFj1Nd66qmnHLVcjLAKJDGPxxNZ4N70AwwAkk3zftJ2OnHiROT14osvKi0trcV7jz76aORYY4waGhradN2BAwfGpCvCnDlzdOLECR0+fFi/+tWv9OUvf1nf//73OxyEE42wCgAAOoXa2trI68CBAyotLdXOnTtb9JPeuXOnSktLdeDAgcixseL3+yOv9PR0WZYV+fzgwYNKTU3Vf/zHfygQCMjj8ai0tFSHDx/W9773PWVkZCglJUU333yz3n333RbXvXgZgGVZeuWVVzRt2jT16tVL1157rd56660r1terVy/5/X5lZmbqK1/5ip5//nn9/Oc/1+rVq1vc87HHHtN1112nXr16adiwYXriiScircTWrl2rp59+Wv/93/8dmaldu3atJGnFihUaNWqUevfuraysLD3wwAO2PijchAesAABAp3Cl3tSX6icdzx7QCxYs0PLlyzVs2DD17dtXFRUVmjx5sn784x/L4/Fo3bp1mjJlig4dOnTZzVmefvppLVu2TD/5yU+0cuVKzZgxQ8eOHVO/fv2iqmfWrFl65JFHtHHjRk2cOFGSlJqaqrVr12rw4MH64IMPNGfOHKWmpmr+/PmaPn26/vCHP2jr1q2RgJueni7p8wd2f/azn2no0KE6cuSIHnjgAc2fP18vvfRSO/9ttQ0zqwAAADHyzDPP6Jvf/KaGDx+ufv36acyYMbrvvvs0cuRIXXvttXr22Wc1fPjwK86U3n333brjjjt0zTXXaMmSJaqpqdHevXujrsflcum6665TWVlZ5L3HH39c48aN05AhQzRlyhQ9+uijeuONNyR93kkmJSVF3bt3j8waN3WXeeihhzRhwgQNGTJE+fn5+tGPfhQ5z07MrAIAgE7h4l85O7GfdG5ubovPa2pq9NRTT2nz5s06ceKEGhoa9Omnn6q8vPyy1xk9enTkn3v37q20tDSdOnWqXTUZY1o06H/99df1s5/9TIcPH1ZNTY0aGhqUlpZ2xeu8++67Wrp0qQ4ePKjq6mo1NDTos88+0/nz523diYyZVQAA0Cn07t27xat5P+nmH5v6STe94l1jc48++qg2bdqkJUuW6D//8z8VCoU0atQo1dfXX/Y6PXr0aPG5ZVkKh8NR19PY2Kj//d//1dChQyVJu3fv1owZMzR58mS9/fbb2r9/vxYtWnTFesrKyvS3f/u3Gj16tH71q19p3759WrVqlSRd8dyOYmYVAAB0Sk39pLOysjR79mytWbNGFRUVjuonvXPnTt19992aNm2apM9nWpv/St5ur732mj7++GPdfvvtkqRdu3bp6quv1qJFiyLHHDt2rMU5brdbjY2NLd7bt2+fwuGwXnjhhchfCuKxBEAirAIAgE6qqZ+02+2WZVkqLCxUfX29o9r0XXvttdq4caOmTJkiy7L0xBNPtGuGtC3Onz+vyspKNTQ06E9/+pM2bdqkn/70p/rBD36gCRMmROopLy9XSUmJbr75Zm3evFmbNm1qcZ0hQ4bo6NGjCoVCyszMVGpqqq655hpduHBBK1eu1JQpU7Rz504VFRXZ8n1cjGUAAACg03J6P+kVK1aob9++GjdunKZMmaJJkybppptusuVeq1ev1qBBgzR8+HD93d/9nT788EO9/vrrLZ7W/+53v6uHH35Yc+fOVU5Ojnbt2qUnnniixXVuv/12ffvb39aECRM0cOBAFRcXa8yYMVqxYoWef/55jRw5Uhs2bNDSpUtt+T4uZpl49nOIg+rqaqWnp+vs2bNtWiwMAADi77PPPtPRo0c1dOhQ9ezZM9HlwAaXG+No8hozqwAAAHAswioAAAAci7AKAAAAxyKsAgAAwLEIqwAAIGHsauOExIvV2NJnFQAAxJ3b7ZbL5dLx48c1cODASK9UdH7GGNXX16uqqkoul0tut7tD1yOsAgCAuHO5XBo6dKhOnDih48ePJ7oc2KBXr17Kzs6O7HjVXoRVAACQEG63W9nZ2WpoaPjC9p7o3Lp166bu3bvHZLacsAoAABLGsiz16NFDPXr0SHQpcChbH7D67W9/qylTpmjw4MGyLEtvvvnmFc957733dNNNN8nj8eiaa67R2rVr7SwRAAAADmZrWK2trdWYMWO0atWqNh1/9OhR3XbbbZowYYJCoZAeeugh3Xvvvfr1r39tZ5kAAABwKFuXAXznO9/Rd77znTYfX1RUpKFDh+qFF16QJF1//fUqLS3VT3/6U02aNKnVc+rq6lRXVxf5vLq6umNFAwAAwDEc1Wd19+7dmjhxYov3Jk2apN27d1/ynKVLlyo9PT3yysrKsrtMAAAAxImjwmplZaUyMjJavJeRkaHq6mp9+umnrZ6zcOFCnT17NvKqqKiIR6kAAACIg07fDcDj8cjj8SS6DAAAANjAUTOrfr9fJ0+ebPHeyZMnlZaWJq/Xm6CqAAAAkCiOCqt5eXnavn17i/e2bdumvLy8BFUEAACARLI1rNbU1CgUCikUCkn6vDVVKBRSeXm5pM/Xm86cOTNy/P33368jR45o/vz5OnjwoF566SW98cYbevjhh+0sEwAAAA5la1gNBoO68cYbdeONN0qS5s2bpxtvvFFPPvmkJOnEiROR4CpJQ4cO1ebNm7Vt2zaNGTNGL7zwgl555ZVLtq0CAABAcrOMMSbRRcRSdXW10tPTdfbsWaWlpSW6HAAAAFwkmrzmqDWrAAAAQHOEVQAAADgWYRUAAACORVgFAACAYxFWAQAA4FiEVQAAADgWYRUAAACORVgFEFPBYFD5+fkKBoOJLgUAkAQIq4AIWLG0bt067dixQ+vXr090KQCAJNA90QUATtA8YOXm5ia6HFvV1tbG/Jrl5eU6c+aMLMtSSUmJJKm4uFgFBQUyxqh///7Kzs6O2f169+4ds2sBAJyN7VbhWHaEquaaB6xp06apqqpKAwcO1KZNm2wJWBdLVOCyLCsh942lJPuxBQBdTjR5jZlVOFZKSkrc71lVVaXx48fH5V4ELgAAroywCnQxNTU1tlw3FAq1GvRLS0uVk5Njyz0BAMmPsArHsitUNdcVA5Zdyw+8Xq8kyeVyKRwORz56vV7WmAIA2o2wCseKR8AhYMWOz+eT3+9XVlaWZs+erTVr1qiiokI+ny/RpQEAOjHCKro0AlbsZGZmqqysTG63W5ZlqbCwUPX19fJ4PIkuDQDQidENAF1eXV1dJGAZYwhYAADYjG4AQBSaB1PLsgiqAAA4CDtYAQnAjlmdD2MGAIlBWAUSgC1JOx/GDAASgzWrQDN27pqVyB2zulpng1iNYzzHrKuNEYCuLZq8RlgFmkmGrUhbk2R/zK+oM45jVxsjAF1bNHmNZQAAAABwLLoBAM3YvWtWV9wxKxFiOY6MGQAkFmEVaMbudYPsmBUfsfx3yZgBQGKxDACIo6YdswKBgIqKihQIBOT3+9kxy8EYMwBILB6wAuKMHbM6H8YMAGKLHawAB2PHrM6HMQOAxGEZAAAAAByLsAoAAADHIqwCAADAsQirAAAAcCzCKoBOKxgMKj8/X8FgMNGlAABsQlgF0GmtW7dOO3bs0Pr16xNdCgDAJrSuAmCr2tramF6vvLxcZ86ckWVZKikpkSQVFxeroKBAxhj1799f2dnZMbsfu1QBQGIRVgHYKiUlxfZ7VFVVafz48bZcO8n2TQGATodlAAAAAHAsZlYB2Kqmpibm1wyFQq3OpJaWlionJyfm9wMAJE5cwuqqVav0k5/8RJWVlRozZoxWrlypW265pdVj165dq3vuuafFex6PR5999lk8SgUQY3as+fR6vZIkl8ulcDgc+ej1elljCgBJxvZlAK+//rrmzZunxYsX6/3339eYMWM0adIknTp16pLnpKWl6cSJE5HXsWPH7C4TDkRbIlyKz+eT3+9XIBBQUVGRAoGA/H6/fD5foksDAMSY7TOrK1as0Jw5cyKzpUVFRdq8ebNeffVVLViwoNVzLMuS3++3uzQ4XPO2RLm5uYkuBw6SmZmpsrIyud1uWZalwsJC1dfXy+PxJLo0AECM2RpW6+vrtW/fPi1cuDDynsvl0sSJE7V79+5LnldTU6Orr75a4XBYN910k5YsWaIbbrih1WPr6upUV1cX+by6ujp23wBaiHULotbEuy1Ra/g1cufQPJhalkVQBYAkZWtYPX36tBobG5WRkdHi/YyMDB08eLDVc770pS/p1Vdf1ejRo3X27FktX75c48aN0x//+EdlZmZ+4filS5fq6aeftqV+tBSPFkStsbMtUWtoVQQAgHM4rnVVXl6eZs6cqZycHN16663auHGjBg4cqJ///OetHr9w4UKdPXs28qqoqIhzxQAAALCLrTOrAwYMULdu3XTy5MkW7588ebLNa1J79OihG2+8UR999FGrX/d4PPz6L07saEHUGtoSAQCAJraGVbfbrUAgoO3bt2vq1KmSpHA4rO3bt2vu3LltukZjY6M++OADTZ482cZK0RbxWstJWyIAANDE9mUA8+bN0+rVq/Xaa6/pwIED+sEPfqDa2tpId4CZM2e2eADrmWee0TvvvKMjR47o/fff19///d/r2LFjuvfee+0uFQ5BWyIAANDE9tZV06dPV1VVlZ588klVVlYqJydHW7dujTx0VV5eLpfrr5n5448/1pw5c1RZWam+ffsqEAho165d+vKXv2x3qXAI2hIBAIAmlkmyR5+rq6uVnp6us2fPKi0tLdHlAAAA4CLR5DXHdQMAAAAAmhBWAQAA4FiEVQAAADgWYRUAAACORVgFAACAYxFWAQAA4FiEVQAAADgWYRUAAACORVgFAACAYxFWAQAA4FiEVQAAADgWYRUAAACORVgFAACAYxFWAQAA4FiEVQAAADgWYRVtFgwGlZ+fr2AwmOhSAABAF0FYRZutW7dOO3bs0Pr16xNdCgAA6CK6J7oAtE9tbW1c7lNeXq4zZ87IsiyVlJRIkoqLi1VQUCBjjPr376/s7Gzb6+jdu7ft9wAAAM5DWO2kUlJSEnbvqqoqjR8/Pq73NMbE9X4AAMAZWAYAAAAAx2JmNQbi9Sv5JuXl5XrnnXdkWZbuuOMOnT59WgMGDFBxcbGMMerXr5+ysrJidr/f//73+uY3v/mF97dt26a8vLyY3QcAAOBilkmy369WV1crPT1dZ8+eVVpaWlzuaVlWXO7jREn2nw8AAIiDaPIaywAAAADgWCwDiIGampq43zMUCrX6kFNpaalycnJifr+6ujq53W5ZliVjjOrr6+XxeGJ+HwAAgOYIqzGQiLZKXq9XkuRyuRQOhyMfvV6vLfXQOgoAACQCywA6KZ/PJ7/fr0AgoKKiIgUCAfn9fvl8vkSXBgAAEDM8YNWJ8at5AADQGUWT11gG0Ik1D6aWZRFUAQBA0mEZAAAAAByLsAoAAADHIqwCAADAsQirAAAAcCzCKgAAAByLsAoAAADHIqwCAADAsQirAAAAcCzCKgAAAByLsAoAAADHiktYXbVqlYYMGaKePXtq7Nix2rt372WP/+Uvf6kRI0aoZ8+eGjVqlLZs2RKPMgEAAOAwtofV119/XfPmzdPixYv1/vvva8yYMZo0aZJOnTrV6vG7du3SHXfcodmzZ2v//v2aOnWqpk6dqj/84Q92lwoAAACHsYwxxs4bjB07VjfffLP++Z//WZIUDoeVlZWlf/iHf9CCBQu+cPz06dNVW1urt99+O/LeV77yFeXk5KioqOiK96uurlZ6errOnj2rtLS02H0jAJCkgsGg5s+fr2XLlik3NzfR5QDoAqLJa7bOrNbX12vfvn2aOHHiX2/ocmnixInavXt3q+fs3r27xfGSNGnSpEseX1dXp+rq6hYvAEDbrVu3Tjt27ND69esTXQoAfEF3Oy9++vRpNTY2KiMjo8X7GRkZOnjwYKvnVFZWtnp8ZWVlq8cvXbpUTz/9dGwKBoBOora2tkPnl5eX68yZM7IsSyUlJZKk4uJiFRQUyBij/v37Kzs7u0P36N27d4fOBwDJ5rAaDwsXLtS8efMin1dXVysrKyuBFQGA/VJSUmJ+zaqqKo0fPz5m17N5lRmALsLWsDpgwAB169ZNJ0+ebPH+yZMn5ff7Wz3H7/dHdbzH45HH44lNwQAAAHAUW8Oq2+1WIBDQ9u3bNXXqVEmfP2C1fft2zZ07t9Vz8vLytH37dj300EOR97Zt26a8vDw7SwWATqWmpqbD1wiFQq3OpJaWlionJ6fD1weAWLB9GcC8efM0a9Ys5ebm6pZbbtGLL76o2tpa3XPPPZKkmTNn6qqrrtLSpUslSf/4j/+oW2+9VS+88IJuu+02lZSUKBgM6uWXX7a7VADoNGKxHtTr9Ur6/MHXcDgc+ej1ellvCsAxbA+r06dPV1VVlZ588klVVlYqJydHW7dujTxEVV5eLpfrr00Jxo0bp1/84hd6/PHH9cMf/lDXXnut3nzzTY0cOdLuUgGgS/H5fPL7/crKytLs2bO1Zs0aVVRUyOfzJbo0AIiwvc9qvNFnFUAyiFfv07q6OrndblmWJWOM6uvreQ4AgO0c02cVANA+8ep96vF4ZFmWJMmyLIIqAMfp9K2rAMAJOtr3VKL3KQC0hrAKADFgR99Tid6nAMAyAAAAADgWM6sAEAOx6Hsq0fsUAC5GWAWAGIjVWlB6nwJASywDAAAHaep9GggEVFRUpEAgIL/fT+9TIEkEg0Hl5+crGAwmupROg5lVAHCQzMxMlZWVRXqfFhYW0vsUSCLN29LZ2UM5mRBWAcBhmgdTep8CncPl2td1tC1dV18CRFgFAADooGjb10XTlq6rt5xjzSoAAAAci5lVAACADrpS+zra0rUfYRUAAKCDrrSulLZ07ccyAAAAAJvRlq79LJNkq3arq6uVnp6us2fPKi0tLdHlAAAASJLq6uoibemMMV26LV00eY1lAAAAAHFAW7r2YRkAAAAAHIuwCgAAAMcirAIAACSpYDCo/Px8BYPBRJfSboRVAACAJLVu3Trt2LFD69evT3Qp7cYDVgAAAJ1MbW3tJb9WXl6uM2fOyLIslZSUSJKKi4tVUFAgY4z69++v7OzsL5zn1H6vhFUAAIBOJiUlJarjq6qqWt1BqzmndjNlGQAAAAAci5lVAACATqampuayXw+FQq3OpJaWlionJ8emquxBWAUAAOhkrrS+1Ov1SpJcLpfC4XDko9frdeza1EthGQAAAECS8fl88vv9CgQCKioqUiAQkN/vl8/nS3RpUbOMU1fTtlM0e80CAAAkq7q6OrndblmWJWOM6uvrHbPFazR5jWUAAAAASah5MLUsyzFBNVosAwAAAIBjEVYBAADgWIRVAAAAOBZhFQAAAI5FWAUAAIBjEVYBAEkrGAwqPz9fwWAw0aUAaCfCKgAgruIZINetW6cdO3Zo/fr1tt8LgD3oswoAiKvmATI3N/eSx9XW1rbr+uXl5Tpz5owsy1JJSYkkqbi4WAUFBTLGqH///srOzo76up1ti0ogWbCDFQCgTdobHqWWAXLatGmqqqrSwIEDtWnTpksGyJSUlI6WHFNJ9r9LIKHYwQoAEHOxDo9VVVUaP358TK8JIPnYtmb1z3/+s2bMmKG0tDT16dNHs2fPVk1NzWXP+frXvy7Lslq87r//frtKBAAkgZqami+8SktLWz22tLS01ePb8gKQGLbNrM6YMUMnTpzQtm3bdOHCBd1zzz0qLCzUL37xi8ueN2fOHD3zzDORz3v16mVXiQCAKHQ0sIVCoVZnUktLS5WTk9Pu67a2ltTr9UqSXC6XwuFw5KPX62XtKdDJ2BJWDxw4oK1bt+p3v/tdZPH8ypUrNXnyZC1fvlyDBw++5Lm9evWS3++3oywAQAd0NOTFM0D6fD75/X5lZWVp9uzZWrNmjSoqKuTz+WJ6HwD2s2UZwO7du9WnT58WT3lOnDhRLpdLe/bsuey5GzZs0IABAzRy5EgtXLhQ58+fv+zxdXV1qq6ubvECADhPU4AMBAIqKipSIBCQ3++3JUBmZmaqrKxMe/bs0X333ac9e/aorKxMmZmZMb8XAHvZMrNaWVn5hR8+3bt3V79+/VRZWXnJ8+68805dffXVGjx4sH7/+9/rscce06FDh7Rx48ZLnrN06VI9/fTTMasdAGCPpgDpdrtlWZYKCwtVX18vj8djy/2aX9eyLNvuA8BeUYXVBQsW6Pnnn7/sMQcOHGh3MYWFhZF/HjVqlAYNGqRvfOMbOnz4sIYPH97qOQsXLtS8efMin1dXVysrK6vdNQAA7EOABBCtqMLqI488orvvvvuyxwwbNkx+v1+nTp1q8X5DQ4P+/Oc/R7UedezYsZKkjz766JJh1ePx8MMOAAAgSUUVVgcOHKiBAwde8bi8vDx98skn2rdvnwKBgCTpN7/5jcLhcCSAtkUoFJIkDRo0KJoyAQAAkCRsecDq+uuv17e//W3NmTNHe/fu1c6dOzV37lx9//vfj3QC+L//+z+NGDFCe/fulSQdPnxYzz77rPbt26eysjK99dZbmjlzpv7mb/5Go0ePtqNMAAAAOJxtmwJs2LBBI0aM0De+8Q1NnjxZ48eP18svvxz5+oULF3To0KHI0/5ut1vvvvuuvvWtb2nEiBF65JFHdPvtt+vf//3f7SoRAAAADmeZJNvsOJq9ZgEAABB/0eQ122ZWAQBA8ggGg8rPz1cwGEx0KehiCKsAAOCK1q1bpx07dmj9+vW234tgjOZs2RQAAAA4U21tbZuPLS8v15kzZ2RZlkpKSiRJxcXFKigokDFG/fv3V3Z2dpuuFc2Wus2DcfPdMNE1sWYVAIAuxLKshNy3pqbmsl9vHoynTZumqqoqDRw4UJs2bYoqGEcTipE40eQ1ZlYBAIDtUlJSoj6nqqpK48ePj+qcJJuDgwirAAB0KVea4bxYKBRqNTCWlpYqJyfniue3J6QCzRFWAQDoQqL9NbnX65UkuVwuhcPhyEev19uma0UTjjsajJGcCKsAAOCSfD6f/H6/srKyNHv2bK1Zs0YVFRXy+XxtOj+acNzRYIzkRFgFAACXlJmZqbKyMrndblmWpcLCQtXX18vj8cT8Xh0NxkhOdAMAAACOUVdXFwnGxhjbgjESi24AAACgU2oeTC3LIqiCHawAAOiM2OUJXQVhFQCATiie258CicQyAAAAEiSarU+l2G5/KrHbEzoHwioAAAkSi4b57dnlqUmSPWPdbsFgUPPnz9eyZcuUm5ub6HJwEZYBAACALo0lFc7GzCoAAAkS7danErs8tUVblld0dEkFSyjih7AKAECCtCfwsMvTlbV3eUU0SypYQhE/LAMAAKATadrlKRAIqKioSIFAQH6/n12ekLTYwQoAgE6GXZ4ur61dFjqypIJZ7I5hBysAAJIYuzxdXluDJEsqOgeWAQAAgC6JJRWdA8sAAABAl8WSisRgGQAAAEAbsKTC+VgGAABAjASDQeXn5ysYDCa6FCBpEFYBAIgRdkICYo9lAAAA/EVbWx4119GdkFrDk+jAXxFWAQD4i/bufHSxaHZCak2SPfsMdAjLAAAAAOBYzKwCAPAXNTU17TqvIzshAbg8wioAAH/R3rWi7IQE2IdlAAAAdBA7IQH2YQcrAABigJ2QgLZjBysAAOKMnZAAe7AMAAAAAI5FWAUAAIBjEVYBAADgWEm3ZrXpebHq6uoEVwIAAIDWNOW0tjznn3Rh9dy5c5KkrKysBFcCAACAyzl37pzS09Mve0zSta4Kh8M6fvy4UlNTZVlWosuxXXV1tbKyslRRUUGrLgdifJyN8XE+xsjZGB/nc+oYGWN07tw5DR48WC7X5VelJt3MqsvlUmZmZqLLiLu0tDRH/UeIlhgfZ2N8nI8xcjbGx/mcOEZXmlFtwgNWAAAAcCzCKgAAAByLsNrJeTweLV68mJ1SHIrxcTbGx/kYI2djfJwvGcYo6R6wAgAAQPJgZhUAAACORVgFAACAYxFWAQAA4FiEVQAAADgWYRUAAACORVjtBFatWqUhQ4aoZ8+eGjt2rPbu3XvZ43/5y19qxIgR6tmzp0aNGqUtW7bEqdKuKZrxWb16tb72ta+pb9++6tu3ryZOnHjF8UTHRPvnp0lJSYksy9LUqVPtLRBRj9Enn3yiBx98UIMGDZLH49F1113HzzkbRTs+L774or70pS/J6/UqKytLDz/8sD777LM4Vdu1/Pa3v9WUKVM0ePBgWZalN99884rnvPfee7rpppvk8Xh0zTXXaO3atbbX2WEGjlZSUmLcbrd59dVXzR//+EczZ84c06dPH3Py5MlWj9+5c6fp1q2bWbZsmfnwww/N448/bnr06GE++OCDOFfeNUQ7PnfeeadZtWqV2b9/vzlw4IC5++67TXp6uvnTn/4U58q7hmjHp8nRo0fNVVddZb72ta+Z733ve/EptouKdozq6upMbm6umTx5siktLTVHjx417733ngmFQnGuvGuIdnw2bNhgPB6P2bBhgzl69Kj59a9/bQYNGmQefvjhOFfeNWzZssUsWrTIbNy40UgymzZtuuzxR44cMb169TLz5s0zH374oVm5cqXp1q2b2bp1a3wKbifCqsPdcsst5sEHH4x83tjYaAYPHmyWLl3a6vEFBQXmtttua/He2LFjzX333WdrnV1VtONzsYaGBpOammpee+01u0rs0tozPg0NDWbcuHHmlVdeMbNmzSKs2izaMfqXf/kXM2zYMFNfXx+vEru0aMfnwQcfNPn5+S3emzdvnvnqV79qa50wbQqr8+fPNzfccEOL96ZPn24mTZpkY2UdxzIAB6uvr9e+ffs0ceLEyHsul0sTJ07U7t27Wz1n9+7dLY6XpEmTJl3yeLRfe8bnYufPn9eFCxfUr18/u8rssto7Ps8884x8Pp9mz54djzK7tPaM0VtvvaW8vDw9+OCDysjI0MiRI7VkyRI1NjbGq+wuoz3jM27cOO3bty+yVODIkSPasmWLJk+eHJeacXmdNSN0T3QBuLTTp0+rsbFRGRkZLd7PyMjQwYMHWz2nsrKy1eMrKyttq7Oras/4XOyxxx7T4MGDv/DDAx3XnvEpLS3VmjVrFAqF4lAh2jNGR44c0W9+8xvNmDFDW7Zs0UcffaQHHnhAFy5c0OLFi+NRdpfRnvG58847dfr0aY0fP17GGDU0NOj+++/XD3/4w3iUjCu4VEaorq7Wp59+Kq/Xm6DKLo+ZVSBBnnvuOZWUlGjTpk3q2bNnosvp8s6dO6e77rpLq1ev1oABAxJdDi4hHA7L5/Pp5ZdfViAQ0PTp07Vo0SIVFRUlujTo84d3lixZopdeeknvv/++Nm7cqM2bN+vZZ59NdGnoxJhZdbABAwaoW7duOnnyZIv3T548Kb/f3+o5fr8/quPRfu0ZnybLly/Xc889p3fffVejR4+2s8wuK9rxOXz4sMrKyjRlypTIe+FwWJLUvXt3HTp0SMOHD7e36C6mPX+GBg0apB49eqhbt26R966//npVVlaqvr5ebrfb1pq7kvaMzxNPPKG77rpL9957ryRp1KhRqq2tVWFhoRYtWiSXizmyRLpURkhLS3PsrKrEzKqjud1uBQIBbd++PfJeOBzW9u3blZeX1+o5eXl5LY6XpG3btl3yeLRfe8ZHkpYtW6Znn31WW7duVW5ubjxK7ZKiHZ8RI0bogw8+UCgUiry++93vasKECQqFQsrKyopn+V1Ce/4MffWrX9VHH30U+YuEJP3P//yPBg0aRFCNsfaMz/nz578QSJv+YmGMsa9YtEmnzQiJfsILl1dSUmI8Ho9Zu3at+fDDD01hYaHp06ePqaysNMYYc9ddd5kFCxZEjt+5c6fp3r27Wb58uTlw4IBZvHgxratsFO34PPfcc8btdpt/+7d/MydOnIi8zp07l6hvIalFOz4XoxuA/aIdo/LycpOammrmzp1rDh06ZN5++23j8/nMj370o0R9C0kt2vFZvHixSU1NNcXFxebIkSPmnXfeMcOHDzcFBQWJ+haS2rlz58z+/fvN/v37jSSzYsUKs3//fnPs2DFjjDELFiwwd911V+T4ptZV//RP/2QOHDhgVq1aResqxMbKlStNdna2cbvd5pZbbjH/9V//FfnarbfeambNmtXi+DfeeMNcd911xu12mxtuuMFs3rw5zhV3LdGMz9VXX20kfeG1ePHi+BfeRUT756c5wmp8RDtGu3btMmPHjjUej8cMGzbM/PjHPzYNDQ1xrrrriGZ8Lly4YJ566ikzfPhw07NnT5OVlWUeeOAB8/HHH8e/8C5gx44drf4/pWlMZs2aZW699dYvnJOTk2PcbrcZNmyY+dd//de41x0tyxjm5QEAAOBMrFkFAACAYxFWAQAA4FiEVQAAADgWYRUAAACORVgFAACAYxFWAQAA4FiEVQAAADgWYRUAAACORVgFAACAYxFWAQAA4FiEVQAAADjW/wPvaUWifeBm9QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 800x300 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "f, ax = plt.subplots(1, 1, figsize=(8, 3))\n",
    "ax.errorbar(train_x_mean, train_y, xerr=(train_x_stdv * 2), fmt=\"k*\", label=\"Train Data\")\n",
    "ax.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "We train the hyperparameters of the resulting distributional GP via type-II gradient descent, as is standard in many settings. We could also do fully Bayesian inference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from qpytorch.models import ExactQEP\n",
    "from qpytorch.kernels import QExponentialSymmetrizedKLKernel, ScaleKernel\n",
    "from qpytorch.means import ConstantMean\n",
    "POWER =1.0\n",
    "\n",
    "class ExactQEPModel(ExactQEP):\n",
    "    def __init__(self, train_x, train_y, likelihood):\n",
    "        super(ExactQEPModel, self).__init__(train_x, train_y, likelihood)\n",
    "        self.power = torch.tensor(POWER)\n",
    "        self.mean_module = ConstantMean()\n",
    "        self.covar_module = ScaleKernel(QExponentialSymmetrizedKLKernel())\n",
    "        \n",
    "    def forward(self, x):\n",
    "        mean_x = self.mean_module(x)\n",
    "        covar_x = self.covar_module(x)\n",
    "        return qpytorch.distributions.MultivariateQExponential(mean_x, covar_x, power=self.power)\n",
    "\n",
    "# initialize likelihood and model\n",
    "likelihood = qpytorch.likelihoods.QExponentialLikelihood(power=torch.tensor(POWER))\n",
    "model = ExactQEPModel(train_x_distributional, train_y, likelihood)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter 1/500 - Loss: 1.597   lengthscale: 0.693   noise: 0.693\n",
      "Iter 2/500 - Loss: 1.589   lengthscale: 0.826   noise: 0.576\n",
      "Iter 3/500 - Loss: 1.508   lengthscale: 0.973   noise: 0.474\n",
      "Iter 4/500 - Loss: 1.459   lengthscale: 1.133   noise: 0.387\n",
      "Iter 5/500 - Loss: 1.418   lengthscale: 1.302   noise: 0.313\n",
      "Iter 6/500 - Loss: 1.357   lengthscale: 1.483   noise: 0.251\n",
      "Iter 7/500 - Loss: 1.290   lengthscale: 1.678   noise: 0.200\n",
      "Iter 8/500 - Loss: 1.232   lengthscale: 1.884   noise: 0.158\n",
      "Iter 9/500 - Loss: 1.181   lengthscale: 2.102   noise: 0.125\n",
      "Iter 10/500 - Loss: 1.127   lengthscale: 2.327   noise: 0.098\n",
      "Iter 11/500 - Loss: 1.066   lengthscale: 2.561   noise: 0.077\n",
      "Iter 12/500 - Loss: 1.006   lengthscale: 2.800   noise: 0.060\n",
      "Iter 13/500 - Loss: 0.952   lengthscale: 3.045   noise: 0.046\n",
      "Iter 14/500 - Loss: 0.904   lengthscale: 3.294   noise: 0.036\n",
      "Iter 15/500 - Loss: 0.857   lengthscale: 3.546   noise: 0.028\n",
      "Iter 16/500 - Loss: 0.807   lengthscale: 3.801   noise: 0.021\n",
      "Iter 17/500 - Loss: 0.758   lengthscale: 4.058   noise: 0.017\n",
      "Iter 18/500 - Loss: 0.715   lengthscale: 4.316   noise: 0.013\n",
      "Iter 19/500 - Loss: 0.677   lengthscale: 4.574   noise: 0.010\n",
      "Iter 20/500 - Loss: 0.641   lengthscale: 4.831   noise: 0.008\n",
      "Iter 21/500 - Loss: 0.605   lengthscale: 5.087   noise: 0.006\n",
      "Iter 22/500 - Loss: 0.572   lengthscale: 5.342   noise: 0.005\n",
      "Iter 23/500 - Loss: 0.544   lengthscale: 5.595   noise: 0.004\n",
      "Iter 24/500 - Loss: 0.521   lengthscale: 5.845   noise: 0.003\n",
      "Iter 25/500 - Loss: 0.501   lengthscale: 6.092   noise: 0.002\n",
      "Iter 26/500 - Loss: 0.483   lengthscale: 6.335   noise: 0.002\n",
      "Iter 27/500 - Loss: 0.467   lengthscale: 6.573   noise: 0.002\n",
      "Iter 28/500 - Loss: 0.455   lengthscale: 6.807   noise: 0.001\n",
      "Iter 29/500 - Loss: 0.445   lengthscale: 7.034   noise: 0.001\n",
      "Iter 30/500 - Loss: 0.437   lengthscale: 7.256   noise: 0.001\n",
      "Iter 31/500 - Loss: 0.431   lengthscale: 7.471   noise: 0.001\n",
      "Iter 32/500 - Loss: 0.425   lengthscale: 7.679   noise: 0.001\n",
      "Iter 33/500 - Loss: 0.420   lengthscale: 7.880   noise: 0.001\n",
      "Iter 34/500 - Loss: 0.416   lengthscale: 8.073   noise: 0.001\n",
      "Iter 35/500 - Loss: 0.414   lengthscale: 8.260   noise: 0.001\n",
      "Iter 36/500 - Loss: 0.411   lengthscale: 8.438   noise: 0.000\n",
      "Iter 37/500 - Loss: 0.409   lengthscale: 8.609   noise: 0.000\n",
      "Iter 38/500 - Loss: 0.407   lengthscale: 8.773   noise: 0.000\n",
      "Iter 39/500 - Loss: 0.405   lengthscale: 8.929   noise: 0.000\n",
      "Iter 40/500 - Loss: 0.404   lengthscale: 9.078   noise: 0.000\n",
      "Iter 41/500 - Loss: 0.402   lengthscale: 9.221   noise: 0.000\n",
      "Iter 42/500 - Loss: 0.401   lengthscale: 9.356   noise: 0.000\n",
      "Iter 43/500 - Loss: 0.400   lengthscale: 9.486   noise: 0.000\n",
      "Iter 44/500 - Loss: 0.400   lengthscale: 9.609   noise: 0.000\n",
      "Iter 45/500 - Loss: 0.399   lengthscale: 9.726   noise: 0.000\n",
      "Iter 46/500 - Loss: 0.398   lengthscale: 9.839   noise: 0.000\n",
      "Iter 47/500 - Loss: 0.398   lengthscale: 9.946   noise: 0.000\n",
      "Iter 48/500 - Loss: 0.397   lengthscale: 10.048   noise: 0.000\n",
      "Iter 49/500 - Loss: 0.397   lengthscale: 10.145   noise: 0.000\n",
      "Iter 50/500 - Loss: 0.396   lengthscale: 10.238   noise: 0.000\n",
      "Iter 51/500 - Loss: 0.396   lengthscale: 10.327   noise: 0.000\n",
      "Iter 52/500 - Loss: 0.395   lengthscale: 10.412   noise: 0.000\n",
      "Iter 53/500 - Loss: 0.395   lengthscale: 10.493   noise: 0.000\n",
      "Iter 54/500 - Loss: 0.395   lengthscale: 10.571   noise: 0.000\n",
      "Iter 55/500 - Loss: 0.394   lengthscale: 10.645   noise: 0.000\n",
      "Iter 56/500 - Loss: 0.394   lengthscale: 10.716   noise: 0.000\n",
      "Iter 57/500 - Loss: 0.394   lengthscale: 10.784   noise: 0.000\n",
      "Iter 58/500 - Loss: 0.394   lengthscale: 10.849   noise: 0.000\n",
      "Iter 59/500 - Loss: 0.393   lengthscale: 10.911   noise: 0.000\n",
      "Iter 60/500 - Loss: 0.393   lengthscale: 10.971   noise: 0.000\n",
      "Iter 61/500 - Loss: 0.393   lengthscale: 11.028   noise: 0.000\n",
      "Iter 62/500 - Loss: 0.393   lengthscale: 11.083   noise: 0.000\n",
      "Iter 63/500 - Loss: 0.392   lengthscale: 11.136   noise: 0.000\n",
      "Iter 64/500 - Loss: 0.392   lengthscale: 11.187   noise: 0.000\n",
      "Iter 65/500 - Loss: 0.392   lengthscale: 11.236   noise: 0.000\n",
      "Iter 66/500 - Loss: 0.392   lengthscale: 11.284   noise: 0.000\n",
      "Iter 67/500 - Loss: 0.392   lengthscale: 11.330   noise: 0.000\n",
      "Iter 68/500 - Loss: 0.391   lengthscale: 11.375   noise: 0.000\n",
      "Iter 69/500 - Loss: 0.391   lengthscale: 11.418   noise: 0.000\n",
      "Iter 70/500 - Loss: 0.391   lengthscale: 11.460   noise: 0.000\n",
      "Iter 71/500 - Loss: 0.391   lengthscale: 11.501   noise: 0.000\n",
      "Iter 72/500 - Loss: 0.391   lengthscale: 11.542   noise: 0.000\n",
      "Iter 73/500 - Loss: 0.390   lengthscale: 11.581   noise: 0.000\n",
      "Iter 74/500 - Loss: 0.390   lengthscale: 11.619   noise: 0.000\n",
      "Iter 75/500 - Loss: 0.390   lengthscale: 11.656   noise: 0.000\n",
      "Iter 76/500 - Loss: 0.390   lengthscale: 11.693   noise: 0.000\n",
      "Iter 77/500 - Loss: 0.390   lengthscale: 11.728   noise: 0.000\n",
      "Iter 78/500 - Loss: 0.390   lengthscale: 11.763   noise: 0.000\n",
      "Iter 79/500 - Loss: 0.389   lengthscale: 11.797   noise: 0.000\n",
      "Iter 80/500 - Loss: 0.389   lengthscale: 11.831   noise: 0.000\n",
      "Iter 81/500 - Loss: 0.389   lengthscale: 11.864   noise: 0.000\n",
      "Iter 82/500 - Loss: 0.389   lengthscale: 11.896   noise: 0.000\n",
      "Iter 83/500 - Loss: 0.389   lengthscale: 11.928   noise: 0.000\n",
      "Iter 84/500 - Loss: 0.388   lengthscale: 11.960   noise: 0.000\n",
      "Iter 85/500 - Loss: 0.388   lengthscale: 11.991   noise: 0.000\n",
      "Iter 86/500 - Loss: 0.388   lengthscale: 12.022   noise: 0.000\n",
      "Iter 87/500 - Loss: 0.388   lengthscale: 12.052   noise: 0.000\n",
      "Iter 88/500 - Loss: 0.387   lengthscale: 12.083   noise: 0.000\n",
      "Iter 89/500 - Loss: 0.387   lengthscale: 12.113   noise: 0.000\n",
      "Iter 90/500 - Loss: 0.387   lengthscale: 12.143   noise: 0.000\n",
      "Iter 91/500 - Loss: 0.387   lengthscale: 12.172   noise: 0.000\n",
      "Iter 92/500 - Loss: 0.386   lengthscale: 12.202   noise: 0.000\n",
      "Iter 93/500 - Loss: 0.386   lengthscale: 12.231   noise: 0.000\n",
      "Iter 94/500 - Loss: 0.386   lengthscale: 12.261   noise: 0.000\n",
      "Iter 95/500 - Loss: 0.386   lengthscale: 12.290   noise: 0.000\n",
      "Iter 96/500 - Loss: 0.385   lengthscale: 12.319   noise: 0.000\n",
      "Iter 97/500 - Loss: 0.385   lengthscale: 12.348   noise: 0.000\n",
      "Iter 98/500 - Loss: 0.385   lengthscale: 12.378   noise: 0.000\n",
      "Iter 99/500 - Loss: 0.384   lengthscale: 12.407   noise: 0.000\n",
      "Iter 100/500 - Loss: 0.384   lengthscale: 12.436   noise: 0.000\n",
      "Iter 101/500 - Loss: 0.384   lengthscale: 12.465   noise: 0.000\n",
      "Iter 102/500 - Loss: 0.383   lengthscale: 12.495   noise: 0.000\n",
      "Iter 103/500 - Loss: 0.383   lengthscale: 12.525   noise: 0.000\n",
      "Iter 104/500 - Loss: 0.383   lengthscale: 12.554   noise: 0.000\n",
      "Iter 105/500 - Loss: 0.382   lengthscale: 12.584   noise: 0.000\n",
      "Iter 106/500 - Loss: 0.382   lengthscale: 12.614   noise: 0.000\n",
      "Iter 107/500 - Loss: 0.382   lengthscale: 12.645   noise: 0.001\n",
      "Iter 108/500 - Loss: 0.381   lengthscale: 12.676   noise: 0.001\n",
      "Iter 109/500 - Loss: 0.381   lengthscale: 12.707   noise: 0.001\n",
      "Iter 110/500 - Loss: 0.381   lengthscale: 12.738   noise: 0.001\n",
      "Iter 111/500 - Loss: 0.380   lengthscale: 12.769   noise: 0.001\n",
      "Iter 112/500 - Loss: 0.380   lengthscale: 12.801   noise: 0.001\n",
      "Iter 113/500 - Loss: 0.380   lengthscale: 12.834   noise: 0.001\n",
      "Iter 114/500 - Loss: 0.379   lengthscale: 12.866   noise: 0.001\n",
      "Iter 115/500 - Loss: 0.379   lengthscale: 12.899   noise: 0.001\n",
      "Iter 116/500 - Loss: 0.379   lengthscale: 12.933   noise: 0.001\n",
      "Iter 117/500 - Loss: 0.378   lengthscale: 12.966   noise: 0.001\n",
      "Iter 118/500 - Loss: 0.378   lengthscale: 13.001   noise: 0.001\n",
      "Iter 119/500 - Loss: 0.378   lengthscale: 13.035   noise: 0.001\n",
      "Iter 120/500 - Loss: 0.378   lengthscale: 13.070   noise: 0.001\n",
      "Iter 121/500 - Loss: 0.377   lengthscale: 13.105   noise: 0.001\n",
      "Iter 122/500 - Loss: 0.377   lengthscale: 13.141   noise: 0.001\n",
      "Iter 123/500 - Loss: 0.377   lengthscale: 13.177   noise: 0.001\n",
      "Iter 124/500 - Loss: 0.377   lengthscale: 13.214   noise: 0.001\n",
      "Iter 125/500 - Loss: 0.377   lengthscale: 13.251   noise: 0.001\n",
      "Iter 126/500 - Loss: 0.377   lengthscale: 13.288   noise: 0.001\n",
      "Iter 127/500 - Loss: 0.377   lengthscale: 13.325   noise: 0.001\n",
      "Iter 128/500 - Loss: 0.376   lengthscale: 13.363   noise: 0.001\n",
      "Iter 129/500 - Loss: 0.376   lengthscale: 13.401   noise: 0.001\n",
      "Iter 130/500 - Loss: 0.376   lengthscale: 13.440   noise: 0.001\n",
      "Iter 131/500 - Loss: 0.376   lengthscale: 13.478   noise: 0.001\n",
      "Iter 132/500 - Loss: 0.376   lengthscale: 13.517   noise: 0.001\n",
      "Iter 133/500 - Loss: 0.376   lengthscale: 13.556   noise: 0.001\n",
      "Iter 134/500 - Loss: 0.376   lengthscale: 13.594   noise: 0.001\n",
      "Iter 135/500 - Loss: 0.376   lengthscale: 13.633   noise: 0.001\n",
      "Iter 136/500 - Loss: 0.376   lengthscale: 13.672   noise: 0.001\n",
      "Iter 137/500 - Loss: 0.375   lengthscale: 13.712   noise: 0.001\n",
      "Iter 138/500 - Loss: 0.375   lengthscale: 13.751   noise: 0.001\n",
      "Iter 139/500 - Loss: 0.375   lengthscale: 13.789   noise: 0.001\n",
      "Iter 140/500 - Loss: 0.375   lengthscale: 13.828   noise: 0.001\n",
      "Iter 141/500 - Loss: 0.375   lengthscale: 13.867   noise: 0.001\n",
      "Iter 142/500 - Loss: 0.375   lengthscale: 13.906   noise: 0.001\n",
      "Iter 143/500 - Loss: 0.375   lengthscale: 13.944   noise: 0.001\n",
      "Iter 144/500 - Loss: 0.375   lengthscale: 13.982   noise: 0.001\n",
      "Iter 145/500 - Loss: 0.375   lengthscale: 14.020   noise: 0.001\n",
      "Iter 146/500 - Loss: 0.375   lengthscale: 14.058   noise: 0.001\n",
      "Iter 147/500 - Loss: 0.374   lengthscale: 14.096   noise: 0.001\n",
      "Iter 148/500 - Loss: 0.374   lengthscale: 14.133   noise: 0.001\n",
      "Iter 149/500 - Loss: 0.374   lengthscale: 14.170   noise: 0.001\n",
      "Iter 150/500 - Loss: 0.374   lengthscale: 14.206   noise: 0.001\n",
      "Iter 151/500 - Loss: 0.374   lengthscale: 14.243   noise: 0.001\n",
      "Iter 152/500 - Loss: 0.374   lengthscale: 14.278   noise: 0.001\n",
      "Iter 153/500 - Loss: 0.374   lengthscale: 14.314   noise: 0.001\n",
      "Iter 154/500 - Loss: 0.374   lengthscale: 14.349   noise: 0.001\n",
      "Iter 155/500 - Loss: 0.374   lengthscale: 14.384   noise: 0.001\n",
      "Iter 156/500 - Loss: 0.374   lengthscale: 14.419   noise: 0.001\n",
      "Iter 157/500 - Loss: 0.374   lengthscale: 14.453   noise: 0.001\n",
      "Iter 158/500 - Loss: 0.374   lengthscale: 14.487   noise: 0.001\n",
      "Iter 159/500 - Loss: 0.373   lengthscale: 14.520   noise: 0.001\n",
      "Iter 160/500 - Loss: 0.373   lengthscale: 14.554   noise: 0.001\n",
      "Iter 161/500 - Loss: 0.373   lengthscale: 14.587   noise: 0.001\n",
      "Iter 162/500 - Loss: 0.373   lengthscale: 14.619   noise: 0.001\n",
      "Iter 163/500 - Loss: 0.373   lengthscale: 14.651   noise: 0.001\n",
      "Iter 164/500 - Loss: 0.373   lengthscale: 14.683   noise: 0.001\n",
      "Iter 165/500 - Loss: 0.373   lengthscale: 14.715   noise: 0.001\n",
      "Iter 166/500 - Loss: 0.373   lengthscale: 14.746   noise: 0.001\n",
      "Iter 167/500 - Loss: 0.373   lengthscale: 14.778   noise: 0.001\n",
      "Iter 168/500 - Loss: 0.373   lengthscale: 14.808   noise: 0.001\n",
      "Iter 169/500 - Loss: 0.373   lengthscale: 14.839   noise: 0.001\n",
      "Iter 170/500 - Loss: 0.373   lengthscale: 14.869   noise: 0.001\n",
      "Iter 171/500 - Loss: 0.373   lengthscale: 14.899   noise: 0.001\n",
      "Iter 172/500 - Loss: 0.373   lengthscale: 14.929   noise: 0.001\n",
      "Iter 173/500 - Loss: 0.373   lengthscale: 14.959   noise: 0.001\n",
      "Iter 174/500 - Loss: 0.373   lengthscale: 14.988   noise: 0.001\n",
      "Iter 175/500 - Loss: 0.372   lengthscale: 15.017   noise: 0.001\n",
      "Iter 176/500 - Loss: 0.372   lengthscale: 15.046   noise: 0.001\n",
      "Iter 177/500 - Loss: 0.372   lengthscale: 15.075   noise: 0.001\n",
      "Iter 178/500 - Loss: 0.372   lengthscale: 15.103   noise: 0.001\n",
      "Iter 179/500 - Loss: 0.372   lengthscale: 15.132   noise: 0.001\n",
      "Iter 180/500 - Loss: 0.372   lengthscale: 15.160   noise: 0.001\n",
      "Iter 181/500 - Loss: 0.372   lengthscale: 15.188   noise: 0.001\n",
      "Iter 182/500 - Loss: 0.372   lengthscale: 15.215   noise: 0.001\n",
      "Iter 183/500 - Loss: 0.372   lengthscale: 15.243   noise: 0.001\n",
      "Iter 184/500 - Loss: 0.372   lengthscale: 15.270   noise: 0.001\n",
      "Iter 185/500 - Loss: 0.372   lengthscale: 15.297   noise: 0.001\n",
      "Iter 186/500 - Loss: 0.372   lengthscale: 15.324   noise: 0.001\n",
      "Iter 187/500 - Loss: 0.372   lengthscale: 15.351   noise: 0.001\n",
      "Iter 188/500 - Loss: 0.372   lengthscale: 15.378   noise: 0.001\n",
      "Iter 189/500 - Loss: 0.372   lengthscale: 15.404   noise: 0.001\n",
      "Iter 190/500 - Loss: 0.372   lengthscale: 15.430   noise: 0.001\n",
      "Iter 191/500 - Loss: 0.372   lengthscale: 15.456   noise: 0.001\n",
      "Iter 192/500 - Loss: 0.372   lengthscale: 15.482   noise: 0.001\n",
      "Iter 193/500 - Loss: 0.372   lengthscale: 15.508   noise: 0.001\n",
      "Iter 194/500 - Loss: 0.372   lengthscale: 15.533   noise: 0.001\n",
      "Iter 195/500 - Loss: 0.372   lengthscale: 15.558   noise: 0.001\n",
      "Iter 196/500 - Loss: 0.372   lengthscale: 15.584   noise: 0.001\n",
      "Iter 197/500 - Loss: 0.372   lengthscale: 15.608   noise: 0.001\n",
      "Iter 198/500 - Loss: 0.371   lengthscale: 15.633   noise: 0.001\n",
      "Iter 199/500 - Loss: 0.371   lengthscale: 15.658   noise: 0.001\n",
      "Iter 200/500 - Loss: 0.371   lengthscale: 15.682   noise: 0.001\n",
      "Iter 201/500 - Loss: 0.371   lengthscale: 15.706   noise: 0.001\n",
      "Iter 202/500 - Loss: 0.371   lengthscale: 15.730   noise: 0.001\n",
      "Iter 203/500 - Loss: 0.371   lengthscale: 15.754   noise: 0.001\n",
      "Iter 204/500 - Loss: 0.371   lengthscale: 15.778   noise: 0.001\n",
      "Iter 205/500 - Loss: 0.371   lengthscale: 15.801   noise: 0.001\n",
      "Iter 206/500 - Loss: 0.371   lengthscale: 15.825   noise: 0.001\n",
      "Iter 207/500 - Loss: 0.371   lengthscale: 15.848   noise: 0.001\n",
      "Iter 208/500 - Loss: 0.371   lengthscale: 15.871   noise: 0.001\n",
      "Iter 209/500 - Loss: 0.371   lengthscale: 15.894   noise: 0.001\n",
      "Iter 210/500 - Loss: 0.371   lengthscale: 15.916   noise: 0.001\n",
      "Iter 211/500 - Loss: 0.371   lengthscale: 15.939   noise: 0.001\n",
      "Iter 212/500 - Loss: 0.371   lengthscale: 15.961   noise: 0.001\n",
      "Iter 213/500 - Loss: 0.371   lengthscale: 15.983   noise: 0.001\n",
      "Iter 214/500 - Loss: 0.371   lengthscale: 16.005   noise: 0.001\n",
      "Iter 215/500 - Loss: 0.371   lengthscale: 16.027   noise: 0.001\n",
      "Iter 216/500 - Loss: 0.371   lengthscale: 16.049   noise: 0.001\n",
      "Iter 217/500 - Loss: 0.371   lengthscale: 16.070   noise: 0.001\n",
      "Iter 218/500 - Loss: 0.371   lengthscale: 16.092   noise: 0.001\n",
      "Iter 219/500 - Loss: 0.371   lengthscale: 16.113   noise: 0.001\n",
      "Iter 220/500 - Loss: 0.371   lengthscale: 16.134   noise: 0.001\n",
      "Iter 221/500 - Loss: 0.371   lengthscale: 16.155   noise: 0.001\n",
      "Iter 222/500 - Loss: 0.371   lengthscale: 16.176   noise: 0.001\n",
      "Iter 223/500 - Loss: 0.371   lengthscale: 16.197   noise: 0.001\n",
      "Iter 224/500 - Loss: 0.371   lengthscale: 16.217   noise: 0.001\n",
      "Iter 225/500 - Loss: 0.371   lengthscale: 16.238   noise: 0.001\n",
      "Iter 226/500 - Loss: 0.371   lengthscale: 16.258   noise: 0.001\n",
      "Iter 227/500 - Loss: 0.371   lengthscale: 16.278   noise: 0.001\n",
      "Iter 228/500 - Loss: 0.371   lengthscale: 16.298   noise: 0.001\n",
      "Iter 229/500 - Loss: 0.371   lengthscale: 16.318   noise: 0.001\n",
      "Iter 230/500 - Loss: 0.371   lengthscale: 16.338   noise: 0.001\n",
      "Iter 231/500 - Loss: 0.371   lengthscale: 16.358   noise: 0.001\n",
      "Iter 232/500 - Loss: 0.371   lengthscale: 16.377   noise: 0.001\n",
      "Iter 233/500 - Loss: 0.371   lengthscale: 16.397   noise: 0.001\n",
      "Iter 234/500 - Loss: 0.371   lengthscale: 16.416   noise: 0.001\n",
      "Iter 235/500 - Loss: 0.371   lengthscale: 16.435   noise: 0.001\n",
      "Iter 236/500 - Loss: 0.370   lengthscale: 16.454   noise: 0.001\n",
      "Iter 237/500 - Loss: 0.370   lengthscale: 16.473   noise: 0.001\n",
      "Iter 238/500 - Loss: 0.370   lengthscale: 16.492   noise: 0.001\n",
      "Iter 239/500 - Loss: 0.370   lengthscale: 16.510   noise: 0.001\n",
      "Iter 240/500 - Loss: 0.370   lengthscale: 16.529   noise: 0.001\n",
      "Iter 241/500 - Loss: 0.370   lengthscale: 16.547   noise: 0.001\n",
      "Iter 242/500 - Loss: 0.370   lengthscale: 16.566   noise: 0.001\n",
      "Iter 243/500 - Loss: 0.370   lengthscale: 16.584   noise: 0.001\n",
      "Iter 244/500 - Loss: 0.370   lengthscale: 16.602   noise: 0.001\n",
      "Iter 245/500 - Loss: 0.370   lengthscale: 16.620   noise: 0.001\n",
      "Iter 246/500 - Loss: 0.370   lengthscale: 16.638   noise: 0.001\n",
      "Iter 247/500 - Loss: 0.370   lengthscale: 16.655   noise: 0.001\n",
      "Iter 248/500 - Loss: 0.370   lengthscale: 16.673   noise: 0.001\n",
      "Iter 249/500 - Loss: 0.370   lengthscale: 16.691   noise: 0.001\n",
      "Iter 250/500 - Loss: 0.370   lengthscale: 16.708   noise: 0.001\n",
      "Iter 251/500 - Loss: 0.370   lengthscale: 16.725   noise: 0.001\n",
      "Iter 252/500 - Loss: 0.370   lengthscale: 16.742   noise: 0.001\n",
      "Iter 253/500 - Loss: 0.370   lengthscale: 16.759   noise: 0.001\n",
      "Iter 254/500 - Loss: 0.370   lengthscale: 16.776   noise: 0.001\n",
      "Iter 255/500 - Loss: 0.370   lengthscale: 16.793   noise: 0.001\n",
      "Iter 256/500 - Loss: 0.370   lengthscale: 16.810   noise: 0.001\n",
      "Iter 257/500 - Loss: 0.370   lengthscale: 16.827   noise: 0.001\n",
      "Iter 258/500 - Loss: 0.370   lengthscale: 16.843   noise: 0.001\n",
      "Iter 259/500 - Loss: 0.370   lengthscale: 16.860   noise: 0.001\n",
      "Iter 260/500 - Loss: 0.370   lengthscale: 16.876   noise: 0.001\n",
      "Iter 261/500 - Loss: 0.370   lengthscale: 16.892   noise: 0.001\n",
      "Iter 262/500 - Loss: 0.370   lengthscale: 16.908   noise: 0.001\n",
      "Iter 263/500 - Loss: 0.370   lengthscale: 16.924   noise: 0.001\n",
      "Iter 264/500 - Loss: 0.370   lengthscale: 16.940   noise: 0.001\n",
      "Iter 265/500 - Loss: 0.370   lengthscale: 16.956   noise: 0.001\n",
      "Iter 266/500 - Loss: 0.370   lengthscale: 16.972   noise: 0.001\n",
      "Iter 267/500 - Loss: 0.370   lengthscale: 16.987   noise: 0.001\n",
      "Iter 268/500 - Loss: 0.370   lengthscale: 17.003   noise: 0.001\n",
      "Iter 269/500 - Loss: 0.370   lengthscale: 17.018   noise: 0.001\n",
      "Iter 270/500 - Loss: 0.370   lengthscale: 17.034   noise: 0.001\n",
      "Iter 271/500 - Loss: 0.370   lengthscale: 17.049   noise: 0.001\n",
      "Iter 272/500 - Loss: 0.370   lengthscale: 17.064   noise: 0.001\n",
      "Iter 273/500 - Loss: 0.370   lengthscale: 17.079   noise: 0.001\n",
      "Iter 274/500 - Loss: 0.370   lengthscale: 17.094   noise: 0.001\n",
      "Iter 275/500 - Loss: 0.370   lengthscale: 17.109   noise: 0.001\n",
      "Iter 276/500 - Loss: 0.370   lengthscale: 17.124   noise: 0.001\n",
      "Iter 277/500 - Loss: 0.370   lengthscale: 17.138   noise: 0.001\n",
      "Iter 278/500 - Loss: 0.370   lengthscale: 17.153   noise: 0.001\n",
      "Iter 279/500 - Loss: 0.370   lengthscale: 17.167   noise: 0.001\n",
      "Iter 280/500 - Loss: 0.370   lengthscale: 17.182   noise: 0.001\n",
      "Iter 281/500 - Loss: 0.370   lengthscale: 17.196   noise: 0.001\n",
      "Iter 282/500 - Loss: 0.370   lengthscale: 17.211   noise: 0.001\n",
      "Iter 283/500 - Loss: 0.370   lengthscale: 17.225   noise: 0.001\n",
      "Iter 284/500 - Loss: 0.370   lengthscale: 17.239   noise: 0.001\n",
      "Iter 285/500 - Loss: 0.370   lengthscale: 17.253   noise: 0.001\n",
      "Iter 286/500 - Loss: 0.370   lengthscale: 17.267   noise: 0.001\n",
      "Iter 287/500 - Loss: 0.370   lengthscale: 17.281   noise: 0.001\n",
      "Iter 288/500 - Loss: 0.370   lengthscale: 17.294   noise: 0.001\n",
      "Iter 289/500 - Loss: 0.370   lengthscale: 17.308   noise: 0.001\n",
      "Iter 290/500 - Loss: 0.370   lengthscale: 17.321   noise: 0.001\n",
      "Iter 291/500 - Loss: 0.370   lengthscale: 17.335   noise: 0.001\n",
      "Iter 292/500 - Loss: 0.370   lengthscale: 17.348   noise: 0.001\n",
      "Iter 293/500 - Loss: 0.370   lengthscale: 17.362   noise: 0.001\n",
      "Iter 294/500 - Loss: 0.370   lengthscale: 17.375   noise: 0.001\n",
      "Iter 295/500 - Loss: 0.370   lengthscale: 17.388   noise: 0.001\n",
      "Iter 296/500 - Loss: 0.370   lengthscale: 17.401   noise: 0.001\n",
      "Iter 297/500 - Loss: 0.370   lengthscale: 17.414   noise: 0.001\n",
      "Iter 298/500 - Loss: 0.370   lengthscale: 17.427   noise: 0.001\n",
      "Iter 299/500 - Loss: 0.370   lengthscale: 17.440   noise: 0.001\n",
      "Iter 300/500 - Loss: 0.370   lengthscale: 17.453   noise: 0.001\n",
      "Iter 301/500 - Loss: 0.370   lengthscale: 17.466   noise: 0.001\n",
      "Iter 302/500 - Loss: 0.370   lengthscale: 17.478   noise: 0.001\n",
      "Iter 303/500 - Loss: 0.370   lengthscale: 17.491   noise: 0.001\n",
      "Iter 304/500 - Loss: 0.370   lengthscale: 17.503   noise: 0.001\n",
      "Iter 305/500 - Loss: 0.370   lengthscale: 17.516   noise: 0.001\n",
      "Iter 306/500 - Loss: 0.370   lengthscale: 17.528   noise: 0.001\n",
      "Iter 307/500 - Loss: 0.370   lengthscale: 17.540   noise: 0.001\n",
      "Iter 308/500 - Loss: 0.370   lengthscale: 17.553   noise: 0.001\n",
      "Iter 309/500 - Loss: 0.370   lengthscale: 17.565   noise: 0.001\n",
      "Iter 310/500 - Loss: 0.370   lengthscale: 17.577   noise: 0.001\n",
      "Iter 311/500 - Loss: 0.370   lengthscale: 17.589   noise: 0.001\n",
      "Iter 312/500 - Loss: 0.370   lengthscale: 17.601   noise: 0.001\n",
      "Iter 313/500 - Loss: 0.370   lengthscale: 17.612   noise: 0.001\n",
      "Iter 314/500 - Loss: 0.370   lengthscale: 17.624   noise: 0.001\n",
      "Iter 315/500 - Loss: 0.370   lengthscale: 17.636   noise: 0.001\n",
      "Iter 316/500 - Loss: 0.370   lengthscale: 17.648   noise: 0.001\n",
      "Iter 317/500 - Loss: 0.370   lengthscale: 17.659   noise: 0.001\n",
      "Iter 318/500 - Loss: 0.370   lengthscale: 17.671   noise: 0.001\n",
      "Iter 319/500 - Loss: 0.370   lengthscale: 17.682   noise: 0.001\n",
      "Iter 320/500 - Loss: 0.370   lengthscale: 17.693   noise: 0.001\n",
      "Iter 321/500 - Loss: 0.370   lengthscale: 17.705   noise: 0.001\n",
      "Iter 322/500 - Loss: 0.370   lengthscale: 17.716   noise: 0.001\n",
      "Iter 323/500 - Loss: 0.370   lengthscale: 17.727   noise: 0.001\n",
      "Iter 324/500 - Loss: 0.370   lengthscale: 17.738   noise: 0.001\n",
      "Iter 325/500 - Loss: 0.370   lengthscale: 17.749   noise: 0.001\n",
      "Iter 326/500 - Loss: 0.370   lengthscale: 17.760   noise: 0.001\n",
      "Iter 327/500 - Loss: 0.370   lengthscale: 17.771   noise: 0.001\n",
      "Iter 328/500 - Loss: 0.370   lengthscale: 17.782   noise: 0.001\n",
      "Iter 329/500 - Loss: 0.370   lengthscale: 17.793   noise: 0.001\n",
      "Iter 330/500 - Loss: 0.370   lengthscale: 17.803   noise: 0.001\n",
      "Iter 331/500 - Loss: 0.370   lengthscale: 17.814   noise: 0.001\n",
      "Iter 332/500 - Loss: 0.370   lengthscale: 17.825   noise: 0.001\n",
      "Iter 333/500 - Loss: 0.370   lengthscale: 17.835   noise: 0.001\n",
      "Iter 334/500 - Loss: 0.369   lengthscale: 17.846   noise: 0.001\n",
      "Iter 335/500 - Loss: 0.369   lengthscale: 17.856   noise: 0.001\n",
      "Iter 336/500 - Loss: 0.369   lengthscale: 17.866   noise: 0.001\n",
      "Iter 337/500 - Loss: 0.369   lengthscale: 17.877   noise: 0.001\n",
      "Iter 338/500 - Loss: 0.369   lengthscale: 17.887   noise: 0.001\n",
      "Iter 339/500 - Loss: 0.369   lengthscale: 17.897   noise: 0.001\n",
      "Iter 340/500 - Loss: 0.369   lengthscale: 17.907   noise: 0.001\n",
      "Iter 341/500 - Loss: 0.369   lengthscale: 17.917   noise: 0.001\n",
      "Iter 342/500 - Loss: 0.369   lengthscale: 17.927   noise: 0.001\n",
      "Iter 343/500 - Loss: 0.369   lengthscale: 17.937   noise: 0.001\n",
      "Iter 344/500 - Loss: 0.369   lengthscale: 17.947   noise: 0.001\n",
      "Iter 345/500 - Loss: 0.369   lengthscale: 17.957   noise: 0.001\n",
      "Iter 346/500 - Loss: 0.369   lengthscale: 17.967   noise: 0.001\n",
      "Iter 347/500 - Loss: 0.369   lengthscale: 17.976   noise: 0.001\n",
      "Iter 348/500 - Loss: 0.369   lengthscale: 17.986   noise: 0.001\n",
      "Iter 349/500 - Loss: 0.369   lengthscale: 17.996   noise: 0.001\n",
      "Iter 350/500 - Loss: 0.369   lengthscale: 18.005   noise: 0.001\n",
      "Iter 351/500 - Loss: 0.369   lengthscale: 18.015   noise: 0.001\n",
      "Iter 352/500 - Loss: 0.369   lengthscale: 18.024   noise: 0.001\n",
      "Iter 353/500 - Loss: 0.369   lengthscale: 18.033   noise: 0.001\n",
      "Iter 354/500 - Loss: 0.369   lengthscale: 18.043   noise: 0.001\n",
      "Iter 355/500 - Loss: 0.369   lengthscale: 18.052   noise: 0.001\n",
      "Iter 356/500 - Loss: 0.369   lengthscale: 18.061   noise: 0.001\n",
      "Iter 357/500 - Loss: 0.369   lengthscale: 18.070   noise: 0.001\n",
      "Iter 358/500 - Loss: 0.369   lengthscale: 18.079   noise: 0.001\n",
      "Iter 359/500 - Loss: 0.369   lengthscale: 18.088   noise: 0.001\n",
      "Iter 360/500 - Loss: 0.369   lengthscale: 18.097   noise: 0.001\n",
      "Iter 361/500 - Loss: 0.369   lengthscale: 18.106   noise: 0.001\n",
      "Iter 362/500 - Loss: 0.369   lengthscale: 18.115   noise: 0.001\n",
      "Iter 363/500 - Loss: 0.369   lengthscale: 18.124   noise: 0.001\n",
      "Iter 364/500 - Loss: 0.369   lengthscale: 18.133   noise: 0.001\n",
      "Iter 365/500 - Loss: 0.369   lengthscale: 18.142   noise: 0.001\n",
      "Iter 366/500 - Loss: 0.369   lengthscale: 18.150   noise: 0.001\n",
      "Iter 367/500 - Loss: 0.369   lengthscale: 18.159   noise: 0.001\n",
      "Iter 368/500 - Loss: 0.369   lengthscale: 18.167   noise: 0.001\n",
      "Iter 369/500 - Loss: 0.369   lengthscale: 18.176   noise: 0.001\n",
      "Iter 370/500 - Loss: 0.369   lengthscale: 18.184   noise: 0.001\n",
      "Iter 371/500 - Loss: 0.369   lengthscale: 18.193   noise: 0.001\n",
      "Iter 372/500 - Loss: 0.369   lengthscale: 18.201   noise: 0.001\n",
      "Iter 373/500 - Loss: 0.369   lengthscale: 18.210   noise: 0.001\n",
      "Iter 374/500 - Loss: 0.369   lengthscale: 18.218   noise: 0.001\n",
      "Iter 375/500 - Loss: 0.369   lengthscale: 18.226   noise: 0.001\n",
      "Iter 376/500 - Loss: 0.369   lengthscale: 18.234   noise: 0.001\n",
      "Iter 377/500 - Loss: 0.369   lengthscale: 18.243   noise: 0.001\n",
      "Iter 378/500 - Loss: 0.369   lengthscale: 18.251   noise: 0.001\n",
      "Iter 379/500 - Loss: 0.369   lengthscale: 18.259   noise: 0.001\n",
      "Iter 380/500 - Loss: 0.369   lengthscale: 18.267   noise: 0.001\n",
      "Iter 381/500 - Loss: 0.369   lengthscale: 18.275   noise: 0.001\n",
      "Iter 382/500 - Loss: 0.369   lengthscale: 18.283   noise: 0.001\n",
      "Iter 383/500 - Loss: 0.369   lengthscale: 18.290   noise: 0.001\n",
      "Iter 384/500 - Loss: 0.369   lengthscale: 18.298   noise: 0.001\n",
      "Iter 385/500 - Loss: 0.369   lengthscale: 18.306   noise: 0.001\n",
      "Iter 386/500 - Loss: 0.369   lengthscale: 18.314   noise: 0.001\n",
      "Iter 387/500 - Loss: 0.369   lengthscale: 18.321   noise: 0.001\n",
      "Iter 388/500 - Loss: 0.369   lengthscale: 18.329   noise: 0.001\n",
      "Iter 389/500 - Loss: 0.369   lengthscale: 18.337   noise: 0.001\n",
      "Iter 390/500 - Loss: 0.369   lengthscale: 18.344   noise: 0.001\n",
      "Iter 391/500 - Loss: 0.369   lengthscale: 18.352   noise: 0.001\n",
      "Iter 392/500 - Loss: 0.369   lengthscale: 18.359   noise: 0.001\n",
      "Iter 393/500 - Loss: 0.369   lengthscale: 18.367   noise: 0.001\n",
      "Iter 394/500 - Loss: 0.369   lengthscale: 18.374   noise: 0.001\n",
      "Iter 395/500 - Loss: 0.369   lengthscale: 18.381   noise: 0.001\n",
      "Iter 396/500 - Loss: 0.369   lengthscale: 18.389   noise: 0.001\n",
      "Iter 397/500 - Loss: 0.369   lengthscale: 18.396   noise: 0.001\n",
      "Iter 398/500 - Loss: 0.369   lengthscale: 18.403   noise: 0.001\n",
      "Iter 399/500 - Loss: 0.369   lengthscale: 18.410   noise: 0.001\n",
      "Iter 400/500 - Loss: 0.369   lengthscale: 18.417   noise: 0.001\n",
      "Iter 401/500 - Loss: 0.369   lengthscale: 18.424   noise: 0.001\n",
      "Iter 402/500 - Loss: 0.369   lengthscale: 18.431   noise: 0.001\n",
      "Iter 403/500 - Loss: 0.369   lengthscale: 18.438   noise: 0.001\n",
      "Iter 404/500 - Loss: 0.369   lengthscale: 18.445   noise: 0.001\n",
      "Iter 405/500 - Loss: 0.369   lengthscale: 18.452   noise: 0.001\n",
      "Iter 406/500 - Loss: 0.369   lengthscale: 18.459   noise: 0.001\n",
      "Iter 407/500 - Loss: 0.369   lengthscale: 18.466   noise: 0.001\n",
      "Iter 408/500 - Loss: 0.369   lengthscale: 18.473   noise: 0.001\n",
      "Iter 409/500 - Loss: 0.369   lengthscale: 18.480   noise: 0.001\n",
      "Iter 410/500 - Loss: 0.369   lengthscale: 18.486   noise: 0.001\n",
      "Iter 411/500 - Loss: 0.369   lengthscale: 18.493   noise: 0.001\n",
      "Iter 412/500 - Loss: 0.369   lengthscale: 18.500   noise: 0.001\n",
      "Iter 413/500 - Loss: 0.369   lengthscale: 18.506   noise: 0.001\n",
      "Iter 414/500 - Loss: 0.369   lengthscale: 18.513   noise: 0.001\n",
      "Iter 415/500 - Loss: 0.369   lengthscale: 18.519   noise: 0.001\n",
      "Iter 416/500 - Loss: 0.369   lengthscale: 18.526   noise: 0.001\n",
      "Iter 417/500 - Loss: 0.369   lengthscale: 18.532   noise: 0.001\n",
      "Iter 418/500 - Loss: 0.369   lengthscale: 18.539   noise: 0.001\n",
      "Iter 419/500 - Loss: 0.369   lengthscale: 18.545   noise: 0.001\n",
      "Iter 420/500 - Loss: 0.369   lengthscale: 18.551   noise: 0.001\n",
      "Iter 421/500 - Loss: 0.369   lengthscale: 18.558   noise: 0.001\n",
      "Iter 422/500 - Loss: 0.369   lengthscale: 18.564   noise: 0.001\n",
      "Iter 423/500 - Loss: 0.369   lengthscale: 18.570   noise: 0.001\n",
      "Iter 424/500 - Loss: 0.369   lengthscale: 18.576   noise: 0.001\n",
      "Iter 425/500 - Loss: 0.369   lengthscale: 18.582   noise: 0.001\n",
      "Iter 426/500 - Loss: 0.369   lengthscale: 18.589   noise: 0.001\n",
      "Iter 427/500 - Loss: 0.369   lengthscale: 18.595   noise: 0.001\n",
      "Iter 428/500 - Loss: 0.369   lengthscale: 18.601   noise: 0.001\n",
      "Iter 429/500 - Loss: 0.369   lengthscale: 18.607   noise: 0.001\n",
      "Iter 430/500 - Loss: 0.369   lengthscale: 18.613   noise: 0.001\n",
      "Iter 431/500 - Loss: 0.369   lengthscale: 18.619   noise: 0.001\n",
      "Iter 432/500 - Loss: 0.369   lengthscale: 18.624   noise: 0.001\n",
      "Iter 433/500 - Loss: 0.369   lengthscale: 18.630   noise: 0.001\n",
      "Iter 434/500 - Loss: 0.369   lengthscale: 18.636   noise: 0.001\n",
      "Iter 435/500 - Loss: 0.369   lengthscale: 18.642   noise: 0.001\n",
      "Iter 436/500 - Loss: 0.369   lengthscale: 18.648   noise: 0.001\n",
      "Iter 437/500 - Loss: 0.369   lengthscale: 18.653   noise: 0.001\n",
      "Iter 438/500 - Loss: 0.369   lengthscale: 18.659   noise: 0.001\n",
      "Iter 439/500 - Loss: 0.369   lengthscale: 18.665   noise: 0.001\n",
      "Iter 440/500 - Loss: 0.369   lengthscale: 18.670   noise: 0.001\n",
      "Iter 441/500 - Loss: 0.369   lengthscale: 18.676   noise: 0.001\n",
      "Iter 442/500 - Loss: 0.369   lengthscale: 18.682   noise: 0.001\n",
      "Iter 443/500 - Loss: 0.369   lengthscale: 18.687   noise: 0.001\n",
      "Iter 444/500 - Loss: 0.369   lengthscale: 18.693   noise: 0.001\n",
      "Iter 445/500 - Loss: 0.369   lengthscale: 18.698   noise: 0.001\n",
      "Iter 446/500 - Loss: 0.369   lengthscale: 18.703   noise: 0.001\n",
      "Iter 447/500 - Loss: 0.369   lengthscale: 18.709   noise: 0.001\n",
      "Iter 448/500 - Loss: 0.369   lengthscale: 18.714   noise: 0.001\n",
      "Iter 449/500 - Loss: 0.369   lengthscale: 18.720   noise: 0.001\n",
      "Iter 450/500 - Loss: 0.369   lengthscale: 18.725   noise: 0.001\n",
      "Iter 451/500 - Loss: 0.369   lengthscale: 18.730   noise: 0.001\n",
      "Iter 452/500 - Loss: 0.369   lengthscale: 18.735   noise: 0.001\n",
      "Iter 453/500 - Loss: 0.369   lengthscale: 18.741   noise: 0.001\n",
      "Iter 454/500 - Loss: 0.369   lengthscale: 18.746   noise: 0.001\n",
      "Iter 455/500 - Loss: 0.369   lengthscale: 18.751   noise: 0.001\n",
      "Iter 456/500 - Loss: 0.369   lengthscale: 18.756   noise: 0.001\n",
      "Iter 457/500 - Loss: 0.369   lengthscale: 18.761   noise: 0.001\n",
      "Iter 458/500 - Loss: 0.369   lengthscale: 18.766   noise: 0.001\n",
      "Iter 459/500 - Loss: 0.369   lengthscale: 18.771   noise: 0.001\n",
      "Iter 460/500 - Loss: 0.369   lengthscale: 18.776   noise: 0.001\n",
      "Iter 461/500 - Loss: 0.369   lengthscale: 18.781   noise: 0.001\n",
      "Iter 462/500 - Loss: 0.369   lengthscale: 18.786   noise: 0.001\n",
      "Iter 463/500 - Loss: 0.369   lengthscale: 18.791   noise: 0.001\n",
      "Iter 464/500 - Loss: 0.369   lengthscale: 18.796   noise: 0.001\n",
      "Iter 465/500 - Loss: 0.369   lengthscale: 18.801   noise: 0.001\n",
      "Iter 466/500 - Loss: 0.369   lengthscale: 18.806   noise: 0.001\n",
      "Iter 467/500 - Loss: 0.369   lengthscale: 18.810   noise: 0.001\n",
      "Iter 468/500 - Loss: 0.369   lengthscale: 18.815   noise: 0.001\n",
      "Iter 469/500 - Loss: 0.369   lengthscale: 18.820   noise: 0.001\n",
      "Iter 470/500 - Loss: 0.369   lengthscale: 18.825   noise: 0.001\n",
      "Iter 471/500 - Loss: 0.369   lengthscale: 18.829   noise: 0.001\n",
      "Iter 472/500 - Loss: 0.369   lengthscale: 18.834   noise: 0.001\n",
      "Iter 473/500 - Loss: 0.369   lengthscale: 18.838   noise: 0.001\n",
      "Iter 474/500 - Loss: 0.369   lengthscale: 18.843   noise: 0.001\n",
      "Iter 475/500 - Loss: 0.369   lengthscale: 18.848   noise: 0.001\n",
      "Iter 476/500 - Loss: 0.369   lengthscale: 18.852   noise: 0.001\n",
      "Iter 477/500 - Loss: 0.369   lengthscale: 18.857   noise: 0.001\n",
      "Iter 478/500 - Loss: 0.369   lengthscale: 18.861   noise: 0.001\n",
      "Iter 479/500 - Loss: 0.369   lengthscale: 18.866   noise: 0.001\n",
      "Iter 480/500 - Loss: 0.369   lengthscale: 18.870   noise: 0.001\n",
      "Iter 481/500 - Loss: 0.369   lengthscale: 18.874   noise: 0.001\n",
      "Iter 482/500 - Loss: 0.369   lengthscale: 18.879   noise: 0.001\n",
      "Iter 483/500 - Loss: 0.369   lengthscale: 18.883   noise: 0.001\n",
      "Iter 484/500 - Loss: 0.369   lengthscale: 18.887   noise: 0.001\n",
      "Iter 485/500 - Loss: 0.369   lengthscale: 18.892   noise: 0.001\n",
      "Iter 486/500 - Loss: 0.369   lengthscale: 18.896   noise: 0.001\n",
      "Iter 487/500 - Loss: 0.369   lengthscale: 18.900   noise: 0.001\n",
      "Iter 488/500 - Loss: 0.369   lengthscale: 18.904   noise: 0.001\n",
      "Iter 489/500 - Loss: 0.369   lengthscale: 18.909   noise: 0.001\n",
      "Iter 490/500 - Loss: 0.369   lengthscale: 18.913   noise: 0.001\n",
      "Iter 491/500 - Loss: 0.369   lengthscale: 18.917   noise: 0.001\n",
      "Iter 492/500 - Loss: 0.369   lengthscale: 18.921   noise: 0.001\n",
      "Iter 493/500 - Loss: 0.369   lengthscale: 18.925   noise: 0.001\n",
      "Iter 494/500 - Loss: 0.369   lengthscale: 18.929   noise: 0.001\n",
      "Iter 495/500 - Loss: 0.369   lengthscale: 18.933   noise: 0.001\n",
      "Iter 496/500 - Loss: 0.369   lengthscale: 18.937   noise: 0.001\n",
      "Iter 497/500 - Loss: 0.369   lengthscale: 18.941   noise: 0.001\n",
      "Iter 498/500 - Loss: 0.369   lengthscale: 18.945   noise: 0.001\n",
      "Iter 499/500 - Loss: 0.369   lengthscale: 18.949   noise: 0.001\n",
      "Iter 500/500 - Loss: 0.369   lengthscale: 18.953   noise: 0.001\n"
     ]
    }
   ],
   "source": [
    "# this is for running the notebook in our testing framework\n",
    "import os\n",
    "smoke_test = ('CI' in os.environ)\n",
    "training_iter = 2 if smoke_test else 500\n",
    "\n",
    "\n",
    "# Find optimal model hyperparameters\n",
    "model.train()\n",
    "likelihood.train()\n",
    "\n",
    "# Use the adam optimizer\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=0.25)  # Includes QExponentialLikelihood parameters\n",
    "\n",
    "# \"Loss\" for QEPs - the marginal log likelihood\n",
    "mll = qpytorch.mlls.ExactMarginalLogLikelihood(likelihood, model)\n",
    "\n",
    "for i in range(training_iter):\n",
    "    # Zero gradients from previous iteration\n",
    "    optimizer.zero_grad()\n",
    "    # Output from model\n",
    "    output = model(train_x_distributional)\n",
    "    # Calc loss and backprop gradients\n",
    "    loss = -mll(output, train_y)\n",
    "    loss.backward()\n",
    "    print('Iter %d/%d - Loss: %.3f   lengthscale: %.3f   noise: %.3f' % (\n",
    "        i + 1, training_iter, loss.item(),\n",
    "        model.covar_module.base_kernel.lengthscale.item(),\n",
    "        model.likelihood.noise.item()\n",
    "    ))\n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Now, we test predictions. For simplicity, we will assume a fixed variance of $0.01.$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAp4AAAEYCAYAAAD1d/bLAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAWVNJREFUeJzt3Xl4VOWhP/DvmX2fyb6HsO+LIGAQxR2sl6pt1dtSC61br6i1tFW5/bXWWoqKrfqoVatWrNViVVCvKIoIKqLsq4QlISGB7Mvs+5z398ckA4EQkpCZbN/P88wzM2fOzHkzJ8l8510lIYQAEREREVGcKXq6AEREREQ0MDB4EhEREVFCMHgSERERUUIweBIRERFRQjB4EhEREVFCMHgSERERUUIweBIRERFRQjB4EhEREVFCMHgSERERUUIweBIRERFRQsQ1eD733HOYMGECLBYLLBYLCgsL8dFHH8XzkERERETUS0nxXKv9//7v/6BUKjF8+HAIIfDqq69i2bJl2LlzJ8aOHRuvwxIRERFRLxTX4NmW5ORkLFu2DLfccksiD0tEREREPUyVqANFIhG89dZb8Hg8KCwsbHOfQCCAQCAQuy/LMhobG5GSkgJJkhJVVCIiIiLqICEEXC4XsrOzoVC034sz7sFz7969KCwshN/vh8lkwqpVqzBmzJg29126dCkeeuiheBeJiIiIiLpZRUUFcnNz290n7k3twWAQ5eXlcDgcePvtt/HSSy/h888/bzN8nlrj6XA4kJ+fj4qKClgslngWk4iIiIi6wOl0Ii8vD3a7HVartd19E97H84orrsDQoUPxwgsvnHVfp9MJq9UKh8PB4ElERETUC3UmryV8Hk9ZllvVahIRERHRwBDXPp6LFy/G1Vdfjfz8fLhcLrzxxhvYsGEDPv7443geloiIiIh6obgGz9raWvzkJz9BVVUVrFYrJkyYgI8//hhXXnllPA9LRERERL1QXIPnyy+/HM+XJyIiomayLCMYDPZ0MagfUqvVUCqV3fJaCZvHk4iIiOIjGAyitLQUsiz3dFGon7LZbMjMzDznedUZPImIiPowIQSqqqqgVCqRl5d31gm8iTpDCAGv14va2loAQFZW1jm9HoMnERFRHxYOh+H1epGdnQ2DwdDTxaF+SK/XA4iO3UlPTz+nZnd+LSIiIurDIpEIAECj0fRwSag/a/lSEwqFzul1GDyJiIj6gXPte0fUnu76/WLwJCIiIqKEYPAkIiKiXq2goABPPvlkTxej2/S3n6czGDyJiIioR1RUVOBnP/sZsrOzodFoMGjQIPziF79AQ0NDTxetR/3hD3+AJEmQJAkqlQqpqam4+OKL8eSTT3Z62fENGzZAkiTY7fb4FLaTGDyJiIgo4Y4cOYLzzz8fhw8fxr///W8UFxfj+eefx7p161BYWIjGxsYeK1skEunxOVHHjh2LqqoqlJeXY/369bjhhhuwdOlSzJgxAy6Xq0fLdi4YPImIiCjhFi5cCI1Gg08++QSzZs1Cfn4+rr76anz66ac4fvw4fvvb37ba3+Vy4Yc//CGMRiNycnLw7LPPxh4TQuAPf/gD8vPzodVqkZ2djXvuuSf2eCAQwK9//Wvk5OTAaDRi+vTp2LBhQ+zx5cuXw2az4f3338eYMWOg1Wrx0ksvQafTnVZT+Itf/AKXXXZZ7P7GjRtx0UUXQa/XIy8vD/fccw88Hk/s8draWsydOxd6vR6DBw/G66+/3qH3R6VSITMzE9nZ2Rg/fjzuvvtufP7559i3bx8effTR2H6vvfYazj//fJjNZmRmZuJHP/pRbM7NsrIyXHrppQCApKQkSJKEBQsWAADWrFmDmTNnwmazISUlBf/1X/+FkpKSDpXtXDB4EhER9SNCAB5Pz1yE6FgZGxsb8fHHH+POO++MzRHZIjMzE/PmzcObb74JcdILLlu2DBMnTsTOnTvxwAMP4Be/+AXWrl0LAHjnnXfwxBNP4IUXXsDhw4fx7rvvYvz48bHn3nXXXfj666+xYsUK7NmzBzfccAPmzJmDw4cPx/bxer149NFH8dJLL+Hbb7/FvHnzYLPZ8M4778T2iUQiePPNNzFv3jwAQElJCebMmYPvf//72LNnD958801s3LgRd911V+w5CxYsQEVFBdavX4+3334bf/vb32LBsLNGjRqFq6++GitXroxtC4VCePjhh7F79268++67KCsri4XLvLy8WPkPHjyIqqoqPPXUUwAAj8eDRYsWYdu2bVi3bh0UCgWuv/76+Nf0il7M4XAIAMLhcPR0UYiIiHoln88n9u/fL3w+nxBCCLdbiGgETPzF7e5Ymb/55hsBQKxatarNx//6178KAKKmpkYIIcSgQYPEnDlzWu1z0003iauvvloIIcRf/vIXMWLECBEMBk97raNHjwqlUimOHz/eavvll18uFi9eLIQQ4pVXXhEAxK5du1rt84tf/EJcdtllsfsff/yx0Gq1oqmpSQghxC233CJuv/32Vs/58ssvhUKhED6fTxw8eFAAEFu2bIk9XlRUJACIJ5544gzvjhAPPvigmDhxYpuP3X///UKv15/xuVu3bhUAhMvlEkIIsX79egEgVuYzqaurEwDE3r1723z81N+zk3Umr7HGk4iIiHqE6GgVKYDCwsLT7hcVFQEAbrjhBvh8PgwZMgS33XYbVq1ahXA4DADYu3cvIpEIRowYAZPJFLt8/vnnrZqWNRoNJkyY0OoY8+bNw4YNG1BZWQkAeP3113HNNdfAZrMBAHbv3o3ly5e3et3Zs2dDlmWUlpaiqKgIKpUKU6ZMib3mqFGjYs/vCiFEqzk1t2/fjrlz5yI/Px9msxmzZs0CAJSXl7f7OocPH8YPf/hDDBkyBBaLBQUFBR163rnikplERET9iMEAuN09d+yOGDZsGCRJQlFREa6//vrTHi8qKkJSUhLS0tI69Hp5eXk4ePAgPv30U6xduxZ33nknli1bhs8//xxutxtKpRLbt28/balHk8kUu63X60+bJH3q1KkYOnQoVqxYgf/5n//BqlWrsHz58tjjbrcbd9xxR6v+pC3y8/Nx6NChDpW/M4qKijB48GAA0eby2bNnY/bs2Xj99deRlpaG8vJyzJ49G8FgsN3XmTt3LgYNGoQXX3wR2dnZkGUZ48aNO+vzzhWDJxERUT8iSYDR2NOlaF9KSgquvPJK/O1vf8Mvf/nLVv08q6ur8frrr+MnP/lJqyD4zTfftHqNb775BqNHj47d1+v1mDt3LubOnYuFCxdi1KhR2Lt3L8477zxEIhHU1tbioosu6nRZ582bh9dffx25ublQKBS45pprYo9NnjwZ+/fvx7Bhw9p87qhRoxAOh7F9+3ZMnToVQLSvZVenNjpw4ADWrFmDxYsXx+43NDTgkUceQV5eHgBg27ZtrZ7TspRqy9KqANDQ0ICDBw/ixRdfjL0nGzdu7FKZOotN7URERJRwzzzzDAKBAGbPno0vvvgCFRUVWLNmDa688krk5ORgyZIlrfb/6quv8Nhjj+HQoUN49tln8dZbb+EXv/gFgOio9Jdffhn79u3DkSNH8K9//Qt6vR6DBg3CiBEjMG/ePPzkJz/BypUrUVpaii1btmDp0qVYvXr1Wcs5b9487NixA0uWLMEPfvADaLXa2GP3338/Nm3ahLvuugu7du3C4cOH8d5778UGF40cORJz5szBHXfcgc2bN2P79u249dZbTxtQ1ZZwOIzq6mpUVlZi7969ePrppzFr1ixMmjQJv/nNbwBEa1U1Gg2efvppHDlyBO+//z4efvjhVq8zaNAgSJKEDz74AHV1dXC73UhKSkJKSgr+/ve/o7i4GJ999hkWLVp01jJ1i7P2Au1BHFxERETUvvYGffR2ZWVlYv78+SIjI0Oo1WqRl5cn7r77blFfX99qv0GDBomHHnpI3HDDDcJgMIjMzEzx1FNPxR5ftWqVmD59urBYLMJoNIoLLrhAfPrpp7HHg8Gg+P3vfy8KCgqEWq0WWVlZ4vrrrxd79uwRQkQHF1mt1jOWc9q0aQKA+Oyzz057bMuWLeLKK68UJpNJGI1GMWHCBLFkyZLY41VVVeKaa64RWq1W5Ofni3/+859i0KBBZx1cBEAAEEqlUiQnJ4uZM2eKJ554Qvj9/lb7vvHGG6KgoEBotVpRWFgo3n//fQFA7Ny5M7bPH//4R5GZmSkkSRLz588XQgixdu1aMXr0aKHVasWECRPEhg0b2h3w1V2DiyQhOtGzN8GcTiesViscDgcsFktPF4eIiKjX8fv9KC0txeDBg6HT6Xq6ONRPtfd71pm8xqZ2IiIiIkoIBk8iIiIiSggGTyIiIiJKCAZPIiIiIkoIBk8iIiIiSggGTyIiIiJKCAZPIiIiIkoIBk8iIiIiSggGTyIiIiJKCAZPIiIiIkoIVTxffOnSpVi5ciUOHDgAvV6PGTNm4NFHH8XIkSPjeVgiIqIB74m1hxJ6vF9eOaJT+y9YsACvvvoq7rjjDjz//POtHlu4cCH+9re/Yf78+Vi+fHk3lpJ6WlxrPD///HMsXLgQ33zzDdauXYtQKISrrroKHo8nnoclIiKiPiAvLw8rVqyAz+eLbfP7/XjjjTeQn5/fgyWjeIlr8FyzZg0WLFiAsWPHYuLEiVi+fDnKy8uxffv2eB6WiIiI+oDJkycjLy8PK1eujG1buXIl8vPzcd5558W2ybKMpUuXYvDgwdDr9Zg4cSLefvvt2OORSAS33HJL7PGRI0fiqaeeanWsBQsW4LrrrsPjjz+OrKwspKSkYOHChQiFQvH/QSkmrk3tp3I4HACA5OTkNh8PBAIIBAKx+06nMyHlIiIiop7xs5/9DK+88grmzZsHAPjHP/6Bn/70p9iwYUNsn6VLl+Jf//oXnn/+eQwfPhxffPEFfvzjHyMtLQ2zZs2CLMvIzc3FW2+9hZSUFGzatAm33347srKycOONN8ZeZ/369cjKysL69etRXFyMm266CZMmTcJtt92W6B97wEpY8JRlGffeey8uvPBCjBs3rs19li5dioceeihRRSIiIqIe9uMf/xiLFy/G0aNHAQBfffUVVqxYEQuegUAAf/7zn/Hpp5+isLAQADBkyBBs3LgRL7zwAmbNmgW1Wt0qPwwePBhff/01/vOf/7QKnklJSXjmmWegVCoxatQoXHPNNVi3bh2DZwIlLHguXLgQ+/btw8aNG8+4z+LFi7Fo0aLYfafTiby8vEQUj4iIiHpAWloarrnmGixfvhxCCFxzzTVITU2NPV5cXAyv14srr7yy1fOCwWCr5vhnn30W//jHP1BeXg6fz4dgMIhJkya1es7YsWOhVCpj97OysrB37974/GDUpoQEz7vuugsffPABvvjiC+Tm5p5xP61WC61Wm4giERERUS/xs5/9DHfddReAaIA8mdvtBgCsXr0aOTk5rR5ryQwrVqzAr3/9a/zlL39BYWEhzGYzli1bhs2bN7faX61Wt7ovSRJkWe7Wn4XaF9fgKYTA3XffjVWrVmHDhg0YPHhwPA9HREREfdCcOXMQDAYhSRJmz57d6rExY8ZAq9WivLwcs2bNavP5X331FWbMmIE777wztq2kpCSuZaauiWvwXLhwId544w289957MJvNqK6uBgBYrVbo9fp4HpqIiIj6CKVSiaKiotjtk5nNZvz617/GL3/5S8iyjJkzZ8LhcOCrr76CxWLB/PnzMXz4cPzzn//Exx9/jMGDB+O1117D1q1bWeHVC8U1eD733HMAgEsuuaTV9ldeeQULFiyI56GJiIioD7FYLGd87OGHH0ZaWhqWLl2KI0eOwGazYfLkyfjf//1fAMAdd9yBnTt34qabboIkSfjhD3+IO++8Ex999FGiik8dJAkhRE8X4kycTiesViscDke7v5BEREQDld/vR2lpKQYPHgydTtfTxaF+qr3fs87kNa7VTkREREQJweBJRERERAnB4ElERERECcHgSUREREQJweBJRERERAnB4ElERERECcHgSUREREQJweBJRERERAkR15WLiOjMZFmg0RtEvTsAhSRBq1JAo1JAq1I2XyugVvK7IRER9R8MnkQJIIRAoyeIGmcANS4/ap1+1LkCCEXaXzhMIUnQqhXQKKOh1KxTYXSWBUPTTFAqpASVnoiobxNC4I477sDbb7+NpqYm7Ny5E/feey8mTZqEJ5988ozPKygowL333ot77703YWXt7xg8ieLAHQijotGLGqcfta4A6lwBBMNyp19HFgK+YAQ+RAAAda4AjtR5oNcoMSrTjLHZVqSZtd1dfCKihKmursaSJUuwevVqHD9+HOnp6Zg0aRLuvfdeXH755d1yjDVr1mD58uXYsGEDhgwZgtTUVKxcuRJqtbpbXp86jsGTqBvVuwPYVtaEQzUuROT2azPPhS8Ywc5yO3aW25Fh0WFcjgUjMszQqZVxOyYR9X/btm3Dfffdh8ceewznn39+3I9XVlaGCy+8EDabDcuWLcP48eMRCoXw8ccfY+HChThw4EC3HKekpARZWVmYMWNGbFtycnK3vDZ1DjuQEXWDikYv3t15HP/65iiKqpxxDZ2nqnH6sa6oFi99eQRr9lWjotELIRJ3fCLqP/75z39i/fr1eO211xJyvDvvvBOSJGHLli34/ve/jxEjRmDs2LFYtGgRvvnmGwBAeXk5rr32WphMJlgsFtx4442oqamJvcYf/vAHTJo0Ca+99hoKCgpgtVrx3//933C5XACABQsW4O6770Z5eTkkSUJBQQEA4JJLLmnVhF5bW4u5c+dCr9dj8ODBeP31108rr91ux6233oq0tDRYLBZcdtll2L17d4fLAgCyLOOxxx7DsGHDoNVqkZ+fjyVLlsQer6iowI033gibzYbk5GRce+21KCsr6463u1dgjSdRFwkhUFLnxrayJlQ5/D1dHIQiAkVVThRVOWEzqDE+x4pxOVbWghLRaTweT+x2eXk5GhoaIEkSVqxYAQD497//jRtvvBFCCKSkpCA/Px8AYDQau60MjY2NWLNmDZYsWdLm69psNsiyHAudn3/+OcLhMBYuXIibbroJGzZsiO1bUlKCd999Fx988AGamppw44034pFHHsGSJUvw1FNPYejQofj73/+OrVu3Qqls+3/iggULUFlZifXr10OtVuOee+5BbW1tq31uuOEG6PV6fPTRR7BarXjhhRdw+eWX49ChQ7Ea1PbKAgCLFy/Giy++iCeeeAIzZ85EVVVVrGY3FAph9uzZKCwsxJdffgmVSoU//elPmDNnDvbs2QONRtMdb32PYvAk6qRwREZRlQvbjzaiyRvq6eK0ye4N4cvD9dhc2ogx2RZMzkuC1cC+TEQUZTKZ2n28rq4OM2fOPG17d7amFBcXQwiBUaNGnXGfdevWYe/evSgtLUVeXh6AaK3s2LFjsXXrVkydOhVAtBZx+fLlMJvNAICbb74Z69atw5IlS2C1WmE2m6FUKpGZmdnmcQ4dOoSPPvoIW7Zsib3myy+/jNGjR8f22bhxI7Zs2YLa2lpotdG+9Y8//jjeffddvP3227j99tvPWhaXy4WnnnoKzzzzDObPnw8AGDp0aOy9fvPNNyHLMl566SVIUnQA6SuvvAKbzYYNGzbgqquu6sI73bsweBJ1kDsQxv5KJ3ZVNMETiPR0cTokGJaxq9yOPRUODE03YnJ+ErJt+p4uFhFRh0JsUVER8vLyYqETAMaMGQObzYaioqJYSCwoKIgFPQDIyso6rbbybMdRqVSYMmVKbNuoUaNgs9li93fv3g23242UlJRWz/X5fCgpKYndb68sRUVFCAQCZxw0tXv3bhQXF7d6PgD4/f5Wx+jLGDyJ2iCEQL07iEq7D1UOH47b/XD64le7WXFoL/7vxWWYe9tvkDdifLe/viwEDte4cbjGjSyrDlMGJWFomgkKTslENCC53e5W93ft2tVmDefGjRsxadKkuJRh+PDhkCSpWwYQnTo6XZIkyHLnZxJpj9vtRlZWVqsm/hYnB9T2yqLXt//F3+12Y8qUKW32L01LS+t8oXshBk8iRGsGa5x+HG8OmlUOPwKh7v2n1Z6ta99D8e7N2Pbpe3EJniercvjxwZ4qWPVqTMq3YVy2FRoVxxkSDSSn9qlsCUQKhQKyLMeu9Xp9t/brPFlycjJmz56NZ599Fvfcc89px7Hb7Rg9ejQqKipQUVERq/Xcv38/7HY7xowZ021lGTVqFMLhMLZv3x6rRT148CDsdntsn8mTJ6O6uhoqlSo2QKmzhg8fDr1ej3Xr1uHWW2897fHJkyfjzTffRHp6OiwWS5eO0dsxeFK/EAhH0OQJodEThN0bhCsQRrQVR0AIQADN19GmnRPbBDyBCOpcAcjd2Hcp4POedZ+m2kp4nXZAkrBzw2oAwI71qzHx4qsBIWCw2JCUnn3W19HqDV0qo8MXwucH6/DNkQYMSTUh26ZDllWPVJMm1reIiAaG9PR0ZGZmIi8vD7fccgtefvllVFRUID09Pa7HffbZZ3HhhRdi2rRp+OMf/4gJEyYgHA5j7dq1eO6557B//36MHz8e8+bNw5NPPolwOIw777wTs2bN6tbpnkaOHIk5c+bgjjvuwHPPPQeVSoV77723VQ3lFVdcgcLCQlx33XV47LHHMGLECFRWVmL16tW4/vrrO1QenU6H+++/H/fddx80Gg0uvPBC1NXV4dtvv8Utt9yCefPmYdmyZbj22mvxxz/+Ebm5uTh69ChWrlyJ++67D7m5ud32M/cUBk/qM4QQcPrCaPQG0eQNoskTbA6aIbgD4Z4uXiuLrz2vS8/zOBrxzKIfdeo5f/3kYJeO1SIQkmOj4QFAo1Ig06JDlk2HbKsemVYdR8YT9XO5ubkoKyuDRhP94nn77bcjGAzGBtHEy5AhQ7Bjxw4sWbIEv/rVr1BVVYW0tDRMmTIFzz33HCRJwnvvvYe7774bF198MRQKBebMmYOnn36628vyyiuv4NZbb8WsWbOQkZGBP/3pT/jd734Xe1ySJHz44Yf47W9/i5/+9Keoq6tDZmYmLr74YmRkZHT4OL/73e+gUqnw+9//HpWVlcjKysLPf/5zAIDBYMAXX3yB+++/H9/73vfgcrmQk5ODyy+/vN/UgEqiF0/453Q6YbVa4XA4+s0bTmd3au1lY3PItHtDCCdwfsxzseiqkQk71rkGz7ORJCDZqEGWVY8sqw4FqUaYtPzOStRb+P1+lJaWYvDgwdDpdD1dHOqn2vs960xe46cH9ZiILHC8yYc6dyAaMD3Rmsy+MmK8PUvf29mh/Y6XFLVZw3nXX99AztDRbTwj8YQAGtxBNLiD2HfcAUkCcmx6jMw0Y3i6GXoNa0OJiKhjGDwpoYJhGWUNHpTUunGk3tOl9cv7go72u9Roo98aJUmCECJ2rdHqutx3M96EAI41+XCsyYf1B+qQn6LHiAwzhqaZ2CRPRETtYvCkuPMFIyipc6Okzo3yBm+faS5PBJMtBeakVNjSsjB9zg+wec3bsNdVwWRLOfuTewFZCJTVe1FW74VKUYtBqUaMzDBjSJoRaiVHyhMRUWsMnhQXLn8IxbVulNR5cLzJ160jxvsTW1omfvfaeijVakiShMJrbkIkFIKqDy6LFpYFSmrdKKl1Q6NSoCDFiFSTBmadGmadChadGiadCkrOHUpENGAxeFK38QbDOFTjxoEqJ6qdfjBrdszJIVOSpD4ZOk8VDMs4VOPCoZrW2yUJMGpUMOtUsUDaclFIUmwaJ+mU50S3RW9oVAqkm7Wc/J7oFL14rDD1A931+8XgSeckFJFRUufGgSoXjjZ4WbNJ7RIiuvSoOxBGlcPf5dfRqZUYlGLA4FQjClKMHOBEA5pSGf39DwaDZ10Zh6irvN7o/NSnrszUWXENnl988QWWLVuG7du3o6qqCqtWrcJ1110Xz0NSAgghUN7oRVGVCyV17n47QIh6L38ogoPVLhysdkGSEJ3mKcWIwWlGpJs5nQwNLCqVCgaDAXV1dVCr1VAo2L+auo8QAl6vF7W1tbDZbLEvOl0V1+Dp8XgwceJE/OxnP8P3vve9eB6KEqDW6UdRtQuHql29bsJ2GriEACrtflTa/dhU0gCzThULofnJBg5yon5PkiRkZWWhtLQUR48e7eniUD9ls9mQmZl5zq8T1+B59dVX4+qrr47nIShOhBCocwdQafejyu7DcbsPLj/DZnfzOBU4ekAPgymC5IwQTEkRsLLi3Lj8Yew97sDe4w5o1QqMz7FiUp4NZt25NQ8R9WYajQbDhw9HMBjs6aJQP6RWq8+5prNFr+rjGQgEEAgEYvedTmcPlmZgCYZlVDv8qHT4UGn3ocrhZxN6HAgB1BzV4NvNRuzfbETZfj2EfGKQjFItIzk9jKSMEJIzQkjKCCM5I4TkzBCS0sOwJIehYHfGDguEZGwra8KOo3YMzzBhcn4SMq1siqf+SaFQcOUi6vV6VfBcunQpHnrooZ4uxoAQkaP9NMsaPKi0+1DvCnJgUJyEghKKd+uxf7MJRZuNaKxpXfOWlhtEOCTBXqdCJKRA3XEN6o63PbJdrZUxboYbU690YsR5XobQDpKFiPUJzbbpMDk/CUPTTBwZT0SUYAlbq12SpLMOLmqrxjMvL49rtXeTcERGWYMXxbUuHKn3IBBijWa8OBuU2L/FiP3fmHBohwHBwIn2c5VaxvDzvBgz3YMx0z1ISo92YYiEAUe9Co21ajRVq9FYo0JjjRpNNdHb9jo15MiJoGRNDWHK5S5MvdKBjPxQwn/Gvs6iV2NSng3jcizQqpjgiYi6qs+u1a7VaqHVanu6GP1KKCKjtN6D4lo3SvvxEpW9gRwBirYa8fVqK4q2Gls1oVtSwhgz3Y2x0z0YNskLrf7073tKFZCcGUZyZhiY4Gvz9Y8Va7HtUwt2fGaBo16Nz95MxmdvJiN/lA9Tr3TivEtcMJh5jjvC6Qvhi0N1+OZIA8ZkWzCtIBlGba/6l0hE1O/0qhrPU3UmQdMJgXAEpfUeHK5x42iDB6EIm9DjydmgxOY1Vnz9oRX2uhPN6Hkj/Rg73Y0xF3iQMzQQmwi9O4SDEvZvMWLrJxYUbTFCbg65SrWMcYUeTL3KiZFTPOimvuADgl6jxJVjMjA0zdTTRSEi6lN6TY2n2+1GcXFx7H5paSl27dqF5ORk5Ofnx/PQA4InEIbdF0KTJwi7NwS7L4gmb/R+pB+th15xaC/+78VlmHvbb5A3YnxPFwcAIMvA4Z0GbFptxbebTLHgZzBHMG22Axd8x4H03Pg1f6s0AhNmujFhphuuJiV2fGbGlrVWVB3RYvcXZuz+wgxzchjnX+7EtNlOZOSfPtK1N76vPckXjOD9XZWYmGfFxcPToOI0TERE3S6uwXPbtm249NJLY/cXLVoEAJg/fz6WL18ez0P3SeGIjLAsEIzICIVlhCICoYiMUERGICyjyRuEwxuKhktvcMA0m29d+x6Kd2/Gtk/f6/GA5LYrseUTC77+0IqGyhMDgAaP9aHwGjsmXuyGWpPY0G9OimDW9+2Y9X07jpdoseUTC3Z8ZoarUYX1byVj/VvRpvjps52YdIkLemP096Y3va+9ye4KB443+TBnXBbSzOz6Q0TUnRLW1N4VfbGp3eELxZq3W0Jj9CJOv90SLmUZobAYEKPKAz5vh/Zrqq2E12kHJAmvPHQXPI5GGK3J+OmDzwBCwGCxISk9u0OvpdUbulxeOQJUlWlxZK8exXv02L/ZiEgoWhOmM0Rw/pVOFH7HgazBPT933snvbTgk4eB2C7avS8bBbZZYjaxKXYoh40sxepoLn/77Z116X8/l/exLVAoJM4en4rz8pJ4uChFRr9aZvMbg2c0O1biwek9VTxej11p01ciEH/Ovnxzs8L7hoITyQ9GgeWSfAWX7dfB7WneUzBvpR+F37DjvElebg4R6ypnf2wwAPwbwUwDjTtouAeh8+TvzfvYHQ9KMuHJMBgwaDjwiImpLr+njSdRbCQEEfBJ8biWqyzQo/VaPI3v1KD+oQzikALANwH0AHoPWcB4KxvgxZJwPo6d6kDs8cJZX721qAPyl+TIEQBkAGaeHTglAEoDGRBYurrqjH+uROg/+9c1RzB6biUEpxm4uIRHRwMLgSQm19L2dHd73eEkRnln0o9O23/XXN5AzdDSAaJOy16mEx6WC16mC16WMXruV8LlV8HuUePVhDbxuBXxuJXweBfxuJXxuRaz5+VQmWxha/UtoqFqP8y55Fj+6f3GfGB3e0ff2TO9rNGxPRs5QL8YW2jH2AgfS8/payG6tu/qxegIRrNp5HOflJ2HmsFQoOfE8EVGXMHhSQnWmf6BG27z0myRFqyihACDjP38tQCg0Fh6HEkH/uY08VqpkWFNDKBjjQXruAaRkV8KWFsLyP74FADi0830cLbqqU/1Ke6oPZEeP2/K+SpIEIUTs/c0e6kPVEYHjJQYcLzHgk39lIz03iHEXujFhpgt5I7p3SqiuOls/4ZP7B+/csBoAsGP9aky8+OoOncczvY9CADuONuFYkxeXjUpHllXf5Z+BiGigYh/PbsY+nl0nBNBQpUbZfh2OFulRsrce1WUXAcgDcAuAlwFUANgKIDf2PIVCwGCJwGiJNF/LMJgjMJgj0BtlfPTqgwDsJ12aTrp9+kTt56q394G011Xjibu+D1taFqbP+QE2r3kb9roq/PKZd6BU5WDf1ybs/cqEwzsNiIRPJE1bagjjLoxOgl8w1tdj/Vvj3U+4o+dvWLoJFw5LRbKx7eVNiYgGCg4u6kEMnh0XDgFlRXqUfRsNmkeLdHA7Tq2ED8CSImHwmAByhvtgsnphSVHDaI6GTJM1Aq1BhqKdis9ED2jq7cETAMLBIJRqdazWMxIKQaVpHaB8HgWKthix9ysTirYYW9UuK1UC+aN8GDbRh+GTvBg02p+waaR6S/AEAIUkYXSWGYVDU2DWqc/+BCKifojBswcxeLavJczs22TEga1G+L2tO08q1TJyhwVQMNqPQaN9GDTaH1vLvKs6OoUT0LF+pWfTH6cbCgUlHNphiNWENtW2DlkqtYyCMX4Mn+TFsEle5I3wQ3VKDuuuCes7cj7P5Tx25fypFBIm5tkwbXAydOo+0CGYiKgbcVQ79SpNtSrs+9qEbzcZUbzHADlyovnWZAtj6AQfBo3yoWCMH7nDAlB1c81ZV/qVttQEtlxrtLp+GSg7Sq0RGHuBB2Mv8MS6RBTv1qN4lwGHdxvgalSheLcBxbsNwKuARidj8DgfcoYGkJIVQkpWENs+fb9bBvp05Dwk+jyGZYHtR5uwr9KBKflJmDwoCWqufEREdBoGT+p2QgCVR7TYt8mIfV+bcLxY1+rx9LwAxs3wYFyhG/mj/O02kyeayZYCc1Lqaf0fTbaUni5aryFJQGp2CKnZIZx3STWEAOqOa1Gyx4Qje804stcEr0uFg9uMOLitHkA9olM1rQEAbHx/DUr33wprcgipOWbkDE1DckYQenMEWn0EWr0MlfrEl4+uBMWeOo+BkIxNJQ3YfcyO6YNTMC7HyhHwREQnYVN7NxsITe1tNZnKEaD0Wz32fmXC3k0mNNWcaGeVJIGCsT6MK/RgbKE7rmuYd4eO9H+kqLb7W0qITlQ/C8Azp2wXOH3i+rb+BQUAuAC4kDU4G1q9DK1Bhk4vQ2eSkZEXQNbgILKHBGBOirRZtt5wHq16NdLMWmhVCmjVyui1SgFdy+1TtrGWlIj6Ija1U1y1zI245eP34bZPx56vzPh2k7HVwCC1VsbIyV6Mm+HGmOkemGxth4Pe6ORwIkkSQ2enCQB7my+nbj/5WgHgDgBfAygAYAPQMkWRtvmSiqrS9o9msoaRNSSArIJoEM0aHEBGfhAaXc+fR4cvBIevY1+0JAlIMWmRm6RHXpIeOTYD9Br2FyWi/oU1nt2sL9V4dmbQTcvciMGgEq8+vBABbwOAdAAfIRokUqE35WDUVCfGFdoxbJILGm3bv1oDua9kf9PdA30iYSDoVyLgUyDgi17LEUP0vlcBv08BjyO62lRVqRb1lWoIcXpTtqQQSM0OIW+EH5fe0IScoX1vIvxTg2hukoEDl4ioV2KNJ3XI4mvP6+CeqYj202vR8kFfB2BKbKvPrcLO9WHsXN/+q/WF6YaoY+Ix0MdgPnXLmcNt0C+hplyDyiNaVJVGL5WlGngcKtQd06DumAY7PrNg8mVOXD2/HilZ5zZDQiIJAdS7Aqh3BbCr3B4Loi0hNMuqg1HLf+FE1LfwvxadwQgA322+zACwAsACAGGc3mTaou98qFPixHOgj0YnkDcigLwRJ2o0hQBcTUpUHdFiy1oLdq63YMdnFuz+wowZc+248oeNfarrR4uTg+jOcjsAwKJXI8uqQ6ZVhyyrDulmHQczEVGvxqb2btZXm9rlCFB+yIiizRYUbbGi7njrkejZQ7zIGf4ltn4857TX6cwclwCb2k9l1qngC0YQlnvtn+I568mBPscOa/HBy6k4tMMIANAaIrjshiZc/L2mHlt9KV5UCglpZm1zENUjy6aDhRPbE1GcsamdOsiIQzsM2Pe1Cfs3G+E5aXCQUiUwbKIXYwvdGHuBB0npYRw77MXWjznHZXdQKyUMzzBjXI4VOTY9ZFmgwRNEnSuAOncAtU4/6twBBEJyTxe1W/TkgK3c4QH8/JHjOLTDgA9eSsWxYh0+ejUVG9+34aofN+CCqx1Q9pP/hGFZoMrhR5XDj52wA4iOrB+ZacbITDNSTdqeLSARDXj95N8tdYQcASoO63BohwGHdhpQtl+HSOjE9C16UwSjp3owboYbI8/3Qm9sHXo4x+W5y7DoMC7HgpGZZmhVJwaKKJprqtLMrYOBwxdCncuPWlcAda4Aapx+eAJ9r5m4Nxgx2Yt7nynH7i9M+HB5KhoqNXjn6Qx8/k4SvvOzeky8yA2pH7ZSO3whbCltxJbSRqSatRjVHEJZE0pEPYFN7d2stzW111eqo0FzR3SFGZ+r9ajY5IwQxs5wY9wFbgwZ7ztrzU9vmBuxr9GqFRidacHYHAvSzbqzP6EdQghUNPpQVO1Eca0bwXD/qBFNtHAI+OYjKz75Vwrc9ugvff5IH25aVIOswcEeLl38SRKQbdVjZKYZIzLMnLaJiM4J12rvQT0ZPIUAHA0qlO3XxcJmY3XrUKgzRDBskg8jJ3swYooXqdmhflnL0xvkJukxLseK4ekmqOIwMXgoIuNInQcHqp0oq/dC7r1/yr1WwCdhwztJ2PBWMgI+BZRqGXNubsAlNzRBOUCymFIhIT/ZgJGZZgxLN3ESeyLqNAbPHpSo4OnzKFBdpsG33xRh+6dLYLL9GY21hafVaCqUAgWjfRgx2YsRk73IG+kfMB+oPSHVpMGIjGhTps2QuJpgbzCMg9UuHKh2odrhT9hx+wtngxJvPZWBb78xAQDyR/nww19XIyO/d6+y1d30GiUm5towKc/GWlAi6jAGzx70zsef44H772+1nGRXCBGtjXE1qeBqVKGpVoWqMi2qSjWoLtOiqbalf9Y9AJ5uvn4KkkIgIy+I4edFg+bQCV7oDL32FPcLFr0aI5vD5ql9NHtCkyeIomonDlW70OQdWMHpXAgBbFtrwarn0uD3KKHSyPjOgnpcfL0digGWwdRKCWOzrZg8KAlWPfuCEsVbKCLD7g3B7g3C7gvB7g1BIQEGjQpGrbL1tUYZl1a0c8Hg2YN+cuvP8drLL+Ci627G9Xf+v9h2IaKTXfvcSvjcCvg8CvhcSrjsSjTVimjAbFLD3aSCy66Gq0mFUOBMn3ZHAdTDaA3D75mLSLgOWn0KrvufvyMpPQBLihVJ6dkdLjNHpHeeUavE8AwzRmaYkW3Tn/0JPaTeHcCROg9K6tyocfrRe//aew97nQpvPpGBg9ui0y8NHufFf/+qBmk5Ay/EKyQJIzJMmFKQdM79k4kGOlkWsPtCaPIGowHTG0JTc9h0B8Kd+v+sVStg1Khg0EQ/iybl2eJW7o5g8DwHHo+n088pLy9HQ0MDdu1S4Fe/vg7BQB2UyjQkZ61CwKdA0J+OoG8IZLkrnSldAKoBVAL4FsA+AM914XXOjCsJdYxeo8SQVCNGZpqRl2SAoo9N1O3yh2Ih9FiTD5F+PG/ouRIiOvjo/RfSEPApoNHKuObWelw41w5F76poSJiCVAPOH5SMvGR+USXqKE8gjLIGD8rqvTja6InLFHljsi2YPTaz21+3Mxg8z4F0ziNtJERX9Gm5btFyOwTAftKlup1LDYDOB+HOYvBsm1GrRI7NgNwkPXKS9Egxarrh96N38IciKGvwoKTWg7IGD0fHn0FjtQor/pKJ4t3RsDVsohf//atqJGcO3FW6Mq06nD8oCTlJeigkqfkSHaTUX/4+iLpKiOhcumX1HpQ2eFDnCsS9pYnBsxv1zeB52isCyEI0eNoB+M76jKXv7TzrPsdLivDMoh+dtr2zqwgBbGpvYdapoiGzOWwmGQfGNFHhiIzSeg/2HHOgosnL5vhTyDKw6QMrPngxDcGAAlq9jO/8tB4XfMcBtYZv1skkCVBKEhSKaCBVKqLN9Va9GukWHTIsWqSbdUgyqBlSqd/wBsM42uBFWb0HRxu98AUTO9dyXwuenED+FG63u0vP27VrF2bOnHna9rv++npcgqBGG+1vxVWEOs+gUcKsU8OiV8GsUyPVpEGuzQCrYWAOolApFRieYcbwDDOaPEHsOe7A/kon/CFOVA8ACgUw87sOjJzixYq/ZKB0nwGr/paOT1ckY9b3mjDjv+wcwNdMCCAsBHBKNw6XP4xjTSe+dGtUCqSZtUg3R4NohkWL5H7UokD9V0QWqHcHUO3wo9rpR43Tj0ZPkF/YO4HB8xRGo7FLz9ProwNMJEkBIeS4B8GBtoqQUiEh1aTFqd0q2/pbFwJQKSVYdCpYdGqYdWqYdSpY9NFrzlN4ZklGDWaNSMOMoSk4VOPCnmMOTs/ULC0nhIXLjuHr1VasW5EMe70aH7yUhnUrkjHzu3ZcdH0TTFZ2WeiIYFjG8SYfjp8aRk1aZNl0yLHpkW3TQ6ceYNMJUK8ihECTN4RqRzRgVjv9qHcFEGb/+HOSkKb2Z599FsuWLUN1dTUmTpyIp59+GtOmTTvr8/rSqPZjx45h6tSpSMvMxsiLr40FwV8+8w5safGpAu/vqwiZdSoMSjFicKoBecmGVktMUuLUOv3Yc8yBgzUu9gVtFg4BOz6z4LM3k1F7LPo3p9HKmP4dBy75fhOS0gduH9DuIklAqkmLnCQ9cm3RftYGDetKKP4q7T58c6QB1U5/XAYDdbe+1tQe9+D55ptv4ic/+Qmef/55TJ8+HU8++STeeustHDx4EOnp6e0+ty8FTwAIBAIoawrgw73V/TIIxptCkpBl1aEg1YiCVAOnb+ll/KEIDlS7cLTBg0ZPEA5faMA3L8kRYO9XJqxbkYxjxdHfV6VKYMrlTlx2YyPS8wbeFEzxlGzUIMemR25ytEaU681Td/IFI/jycB32Vzn71P82Bs9TTJ8+HVOnTsUzzzwDAJBlGXl5ebj77rvxwAMPtPvcvhY8gd63VntvZ9Aom2s1jRiUYmDTWh8Sisho8gRR7w6i0RNEgyeABncQTv/AC6RCAIe2G/Dpm8koaR4BL0kC4y90Y/JlLmQWBJCaFYrLRPQVh/bi/15cds6LVvRFRq0SGRZdrJ9ohkUHo5a1otQ5QgjsO+7EVyX1CR8Y1B36WvCM619oMBjE9u3bsXjx4tg2hUKBK664Al9//XU8D029WKpZiyGpRgxJMyLTouOAgj5KrVQg3aJDuqV1zXQwLKPJG0S9OwC7N4RGz4nJkvtr3yhJAkae78XI870oK9Jh3YpkfPu1CXs2mrFnoxkAoFLLSM8LInNQEJkFQWTkB5BZEERK5rkF0q1r30Px7s3Y9ul7Ay54egIRHKnz4EjdiWnnzDpVdAS9ORpEMyw6Lv9JZ1Tr8uOzolpUsS97wsQ1eNbX1yMSiSAjI6PV9oyMDBw4cOC0/QOBAAKBQOy+0+mMZ/EoQVQKCbnJegxONWFwqpFL8PVzGpUi9oF/MiEEnP4wmjzB5pU7oit4NHo6v2pHb1Yw2o9bHqpEVakGX75nw7FDOtRUaBAKKFB5RIfKI63fF5VGRkZeEBmDgkjNDiEpPYTkjBBMNhesqUEo2/gv3VRbCa/TDkgSdm5YDQDYsX41Jl58NSAEDBZbm6uXDYQZL1z+MFx+N0pqT8xQkmbW4rx8G0ZlWqDsYws/9FYRWcAdCEOtlPpk39tAOIJNJQ3YU+GA3F/++fQRveq3ZenSpXjooYd6uhjUDYxaJQpSorWa+clGaFQcST7QSc3zOVr1ahSg9ewR4YgMTyACdzAMbyAMdyAMbzDSfB2GOxCBNxCGLxTp1QH11GbvG++tBRDtC9pYo0b1UQ1qjmpQfVQbvV2uQTiowPESHY6XtNWnOYLoqmVlzZejzdcvn7anx9HY5ty+Jxuoi0XUuQL45NsafF3SgPPykzA+x9ov/ie5A2FIAHRqZbcG6nBEhjsQhssf/Vt0B8Jw+8NwNV+7AyF4gyf+Flv63uYk6ZGbpIf5HPreRmQBhy8ECdHV4rQqRbe3ih2sduGLQ3VwBzgIsCfENXimpqZCqVSipqam1faamhpkZp7eH2Hx4sVYtGhR7L7T6UReXl48i0jdpGVgUH6KAQUpRmRYtGxCpw5TKRWwGhRnnUs1Igt4g2EEwnL0EoqcdjvYcj8cQYM7mNAPlzM1eyuUQGp2CKnZIYwrPNEsLEeAhmo1asqjgbShWo2mWjWaqtWoPSYD0AHIa75cdNKRLgWwAAA/ODvD5Q/ji0N12FzagIm5NkzKs/WpPqH+UATljV6UN3hR3uiFw3di8JpKIUGrVkCrioa1VrdVSqiVEkIRgVBERjAiI9R8CYZlBCMCofCJbaFI577dNXqiLRd7jzsAABa9unkxjmgQtRlOH2TrDpxo/WjyhmK3nb5wqxpIhSRBp1ZAr1FCp1JCp1FCr45edGoFdGolVEoJquZFC1QKBRQKtLpWShKUSgm+YARfHKpDeaO3i2eAukNc/+I0Gg2mTJmCdevW4brrrgMQHVy0bt063HXXXaftr9VqodVq41kk6kZmnQoFKdER6LlJHBhE8adUSNF5WTu4fygi4+uSBuwst59Tc1rAd+YPqq42e7dIyzEgLad1IAUAn8cLj0OFploN7LUaNNVq0FQXva4tvwH2utEAppz2epfd9AGmXlkAWxpH1J9JICRjS2kjdhxtwugsC6YMSorrSmXhiBxdWrSTtZLhiIwqhx9Hm4Nmrct/xhr/sCwQDkTgCfT84BinL4T9vhD2V0a7y5m0KuQk6SEB0ZDpDXZ4ajZZCHiDEXj74KAfaltCplOaP38+XnjhBUybNg1PPvkk/vOf/+DAgQOn9f08FUe19y5qpYScJD0GpRhRkGJE8gBZUpL6vlqnH2uLalDrDJx95zYsumpkN5fohK40fwsB7Nt0EK889F0ACgDySdfbAUxGel4AI6d4Mep8D0ZM9rbZV5SiJAkYmmbC1IJkZFqjXR6EEM01g9HawGBz7WBLjWAoIp9Uwx6J1bSfuD5RAx+WBSQJ0KqitXTR2rroJVqT11yjp47WUNa7Azja4EWl3dfp2kfqWT0xywRHtZ/ipptuQl1dHX7/+9+juroakyZNwpo1a84aOql3UEgSClINGJttRUGKASqu+kN9ULpFhx9OzcfOiiZ8XdLQ5z/MJQnIG2GNrV429aofYON778BeV420XDMqjwjUVmhRW6HFl+8mwWQL4/wrnJh2lROZBcGeLn6vIwRQXOtGca0beo0S4S40N3fkGP5QBP5QBHawNrq/GsizTHRUQlYu6irWePacFJMGY7MtGJVp6VN9oIjOxuENYd2BGhxt6Hg/r/aa2gHgeElRmwN77vrrG8gZOrrd557LSPMzrV7mdSlQvNuAA9sM+PZrE1xNJ/6G80b6Me0qByZf6oLe1PtXZSHqrVr+L5zc3eaVh+6Cx9EIozUZP33wmVbdbeI1qwRrPKnP0qgUGJlhxtgcC7Ks+p4uDlFcWA1qfG9yLoqqnPjiUF2H+o6d7QNDo402z7YEwJZrjVYX1ymMTl4ZTZKk2H2DWcaEmW5MmOlG5K5aHNhqxOaPrdi/2YiKgzpUHNThvRfSMP5CN6Zd5cTw87xQsDGDqFMWX3veGR9ra5aJgTqrxKkYPAc4SQJykwwYk2XB8AwT1GxKpwFidJYFBSlGfH6oDkVV5zZnsMmWEmv2nj7nB9i85m3Y66pgsqV0U2m7TqkCxhZ6MLbQA1eTEtvXmbHlEyuqy7TYud6CnestSEoP4fwrnZh6pROp2WwGJupLDu3bhUfvfgyPPfYYzj///J4uzlmxqb2b9famdkkCkgwaZFh0yLTqMDjFeNYpbIj6u/IGL9YdqIHd2/XQdaZm795ICODYYS02r7Fi5wYzfO4TM1LkDvNjwkUuTLjIjfRchlCiMzm5C05HutvEq/Vjw/LH8P4bL+Oee+7BU089FZdjnA2b2inGpFUhw6pDpiV6SbdoOe0R0SnyUwz48QWDsPlII7YfberS1EtnavbujaKDkwLIG1GLa39eh31fmbDlEwsO7TTgWLEOx4p1+PCVNGQNDmDCRS5MnOlGxqAgODUv0QknB8mOdLfxeRQI+BRQKgUUSgGlEtFrlYBCiQ79fbXVr/Tzj94FAPz73//GjTfeCCEEUlJSMHp0+/3LewprPLtZT9Z4atUKZJh1zbWZ0XWKz2UFCaKBqM4VwLqimgG5drPbrsTeTSbs2WjC4Z0GyJETn4TpeQFMmOnGxIvdyB4SYAglOom9rhpP3PV9WFOzMPaC/8a2df+Bu6kKo6eth6OhAHXHNHDb26/rUyiaA6nqRCBVqQSU6uhtpRqoOrIHQAjAhWctUyLjXWfyGoNnN0tU8NSoFEg3a2NrYmdYtG2uDkFEnSeEwO5jDnxVXN/hia77G49Tgf3fmLD7SxMO7jAgEjrR/zslK4iRU7zIyA8iPTeItLwgbGlhDlCiASUclLDvGyNKv9Wj7pgGtRUyGmuMgFAAEACCAFoviqNUCUTC3fGt7XWcbfUyBs8uYPCMMmiUSDJqWgXNJIOaS1ISxZnLH8KGg3UornX3dFF6lN+jwLebjdiz0YQDW40IBU5PmGqtjLScINLzQkjPCyItNxpK0/OC0Op77ccMUacdL9Fgy8dWbF9ngdd1etc1rSGC9NwQ0nJa/g5CSMsNIjUnCJ0h+rcgRwA5IiHSch2WIEeASERqvg9EwtHt4bCESCh62+cJIBJWIBKWUH/8W6x9479OO/7GjRsxadIkGI3GuL8XLdjHsw/Sa5RIMqhh1WtgM6iRZIheW/Vq9skk6iFmnRpzJ2ajuNaNDQdr4fIPzLXRdUYZUy5zYcplLpTs2Yd3nv4Lckf8EQHvNNRWaFBfqUEooEDlER0qj+hOe77JGoYtLQxrWvTalhaCLbXldhjWlDBUmjOH055YDYboZD63Ajs+M2Pzx1YcO3zid9yaGsKEi9zIHHTiy5Y5KXLWriiK5v6d0RDW2S9m0Sngjh0OYu0bJ/qVKhQKyLIMvV6f0NDZWQyecaSQJOg1J5ZH02uUMDQvi6ZXK2HQqBguifqAYekm5CXrsamkAbsr7GdcL3sg2P3lu6g+ugnDz3sVP/pNHgAgEgEaq9WordCg7pgGNRUa1FVE77sdqtjlWPGZX9dki4bQrMEB5I3wI294ANlDA1BrBFeDoR4hy0DJbj02r7Fiz1cmhIPRmn6lSmBcoRvTZjswcooXih76+G6Zxi0rOweL7v4fvPzyy6ioqEB6enrPFKiD2NTezVqWRGtZh5eI+pdqhx9fH6nH0QZvvwugZ1qhqaMrs5xKqzfA61KgqVYNe50K9joVHPUq2OvUaKpTwVGngr1eFftAb+0oJEUdUjIDsNddh3CoHnpTMhb8/lkolXKbx4znZP00cDTVqrD1Ewu2fGJBY/WJsROZBQFMn+PAlMtcMNnOvvBEIoSDQYwflII547IghEAwGIRWqz37E7sZ+3gSEcWZ3RvEnmMO7K9ywteB1Y/6gkVXjezW1+vISi1CAL+aPQNAHoACAJMAnA/g5L5rEqLNkS3XLeYCWAvA3+HjEbUl6JewZ6MJW9daULzLACGibeU6QwSTL3Nh2mwH8kb0ztkcuGQmEdEAYDNocPGINMwYmoKDNS7sPeYYkFMwnavoB3lD82UXgHfb2Euccq0CsBzAPAAeAGsAvAuPUwGjZWDOQkCdJwRQtl+HLR9bsesLEwLeE62UQyd6MX22AxNmuqHR9dr6uT6JNZ5ERN2k1unH7mMOHKx2IhTptf9az+hMTe1Ax1ZmOVVHm77PdNwzHXP2T/4PzobpKNpihb3uRFOoQiEwdIIP42a4MW6GG0npA3MwGLWvqVaFbZ9asPUTC+orT/z+JGcGMfUqJ6Ze4URyZt/53WGNJxHRAJVu0eHKMTpcNDwV+6uc2HvMgUZPsKeL1WHtBcWOrMzS3cc90zHHTg8hd3gThGjC8WIt9m4yYd8mE6pKtTi8y4DDuwxY9bd05A73Y/wMN8bOcCOrgCsvnapltoCrf/obZOZPRDAgIehXIBw88Uad9vXp1A0SmlfiQWwFntjKPKqTtis6tjJPdxMiOh2Yo16FY8VabFtrweGTmtI1OhmTZrkw9UonBo/zcS7aBGDwJCLqZjq1EpPzkzA5PwkN7gCONnpR3uDFcbuvz05I3zKC1paWhelzfoDNa96Gva4KJltKjx1TkoDc4QHkDg/g6vkNqK9UY98mE/ZuMqLsWz2OHdbh2GEdPno1FUnpIYya6sHoqR4MP8/bb+YWFSLaP9HjVMLjUMLtiF63uu9UwutSIuSPBstgQEIooIDX9TQi4c14+t4vAHw/7mVVKrdA4H4YTEuhN0+GRidDoxXQ6GSotQIarRy9rRPQnrRNfdo+J+4rVQIehxKOehUcDdHBa456FRyNKjibbwfbmHd22EQvpl4VbUrvL78LfQWb2omIEiQiC1TafShv9KK80Ysap79PjYwPB4NQqtWxmsdIKBT3Nem7ekxXkxLffmPEvk0mHNphQPiklZeUahlDx/miQXSaB+l5oV5VGyrL0XkjXU0quJqUcDep4LIr4WxUwW1XwtWkhKspetvtUJ5hVoAzOQqgHtGBWlcDqAWQDuAjSJIMtTYZam0eJCBWRXnae3PSfSG3NRH6md7MewA83Xz9VCfKfO70pgiS0kMYP9Pd55rSz6avNbUzeBIR9RB/KBINoQ1eVDR5EQzLrT7kpZM+4U/eHooI+EP9YyR9IgT9Eor3GFC0xYiiLYZWU+QAQHJGCKOneTBqqgdDx/ugNcjdFkSFAAI+CV5X65pIj/OU2w4lPM37uO3KdsJb25RqGSZrBEZLJHrdfNn43hOIBs1GAG4Aqzv8ml2dJcDv9UJuDqSNVVVwOxyQZQkrHv8feF2N0JuScfX8lxEMSlCqUqDR5iEUUECOaBEKKBDwKxAKSAgGFNFa2kDL/WiNbeikWttQUILJGoElJQxranQxAmtquPX9lHC/HiDE4NmNGDyJiE4nhEC104/DNW4U17rh8IV6ukh9hhBA3TE1irYaUbTFiJK9+lbr0APRPok6YwQ6owx980VnjEBvkqEzyNCboo9JUrRm0udWwu9VxG77PNHbfk/0tpC7lmL1pgjMtghMSWFYkiMw2cIwJ0VgTgpHt9ui24zWCLR60WZYPpcpsroaPLt6TE6H1TV9LXiyjycRUR8jSRKyrHpkWfW4eEQaap1+HK6NhtC+NJipJ0gSmteTt2PW9+wI+CQU7zJEg+hWI5pq1IiEJXgcKngc3XdcpVqG0SLDaI3AZInWRhrMzTWTlgj83h3Y8+XDmPX9BzBs4liYbZF2lxHtqKXv7TxtW1dmKCDqLgyeRER9XLpFh3SLDhcOS0W9O9BcE+pCvZsh9Gy0eoGxhR6MLfREm8W9Cvg8Cvg9iuZaTGXsvs+tgN+rjN0WAtCb5OilpYb01NumCHSG6ICY9prvVz77BiqPbMKxw29h6pXdN5F/WzMGxHOGAuD0sNuTQbdl5P7c237D5VZ7CQZPIqJ+JNWkRapJi8KhKWjyBHGwxoV9xx1w+fvPYIp4kSRAZ5ShM8Zn5oFT5ys9eSnSnRuifS93rF+NiRdf3eZSpN21JGi8Zyg4tZzxDrrt2br2PRTv3oxtn77H4NlLMHgSEfVTSUYNLhiSgmkFySht8GDPMXu/XGO+r1h87Xln3cfjaGyzdhDovj6QtrRM/O619bHZAgqvuSmuMxQkYiquk0N9RwN9vEMvtY2Di4iIBhCHN4S9xx34ttIBbz9ZY76vOJeBPkDfHnwT76m4uvLe9uX382QcXERERL2W1aDGzOGpKByaguJaN3Yfs+N4k6+ni9WrxKtf4EAe6HNyyJQkKe7zv1LvxeBJRDQAKRUSRmaaMTLTjAZ3AHuOO1Bc44Y7wL6g8eoX2BMDfQaK3jSgidrH4ElENMClmLS4dGQ6Lh2ZDk8gjDpXAHXuAGqdAdS5/LD7Qv2uX+i5DvQBumewT08sRdof9aYBTdHjRheAUEiAQnFiBahgWO53f0udxeBJREQxRq0KRq0KBanG2LZgWEa9O4A6VwC1ruh1vTuAiNx3P0HPdaAP0D19BBM90GegSESgN2qVmDE0FcMzTFBI0XXGFJIUDZ1nmDsrFJHR4A5G/57cAdS7Aqh3BwfUSmQMnkRE1C6NSoFsmx7ZNn1sWyAcQUWjF6X1XpTVe9hEfw7Y/7H7xTPQq5USJucn4fyCZGhUirM/odVzFci06pBp1bXa7vSHYiG03h1AQ/MXO7VKAbVSAY0yeq1WStFtihO300zac/6ZEiluo9qXLFmC1atXY9euXdBoNLDb7Z1+DY5qJyLqG2pdfpQ1h9Aqhx9yL29PPLWpHeh8v0D2wRxYJAkYlWnBhcNSYNape7o4vUqvGNUeDAZxww03oLCwEC+//HK8DkNERL1AulmHdLMO0wYnwx+KoLzRi9J6D442eOAJ9L5mRA70oc7ITYouT5th0Z19Z2pX3ILnQw89BABYvnx5vA5BRES9kE6txIgMM0ZkmAEA4YiMiBCQZSAiBCKygCwLhGUBufl+y8XhC6HRE21ubPQEEzrXKAf60KmSDGrMHJ6GYemmni5Kv9Gr+ngGAgEEAoHYfafT2YOlISKi7qBSKrr8YeMLRtDgiYbQBk8Qje4gGj3BuPQp5UAfaqFTKzF9SDIm5tqgVLQ9UIi6plcFz6VLl8ZqSomIiPQaJXI1BuQmtW7q9gTC2Hfcgb3dvA49B/oMLFq1Aja9BjaDGja9GlaDGjaDBqkmDbQqZU8Xr1/qVPB84IEH8Oijj7a7T1FREUaNGtWlwixevBiLFi2K3Xc6ncjLy+vSaxERUf9l1KowfUgKphYko6TOjV0VdhzjCky9kkalQIZFh2BYRp0r0CMDz6x6NbKsOtgMzSHToIZNr4Few3CZaJ0Knr/61a+wYMGCdvcZMmRIlwuj1Wqh1fataQGIiKjnKBQShmeYMTzDjDpXALsr7DhQ7UQo0rtH1fckpUKCUiFB1Xx94rYCKoUEAQF3IAJPINzpuVpVCglpZi0yrDpkmKPTBiUZ1LF5LYNhGdUOP47bfai0+1Dt9CMYlrv155MkINWkRU7zFGDZNh1HofcinQqeaWlpSEtLi1dZiIiIuizNrMUVYzIwc3gqvq10Ys8xO+zeUE8Xq0fo1EpkNc8XmWXVIc2shVoZDZZnmtz8VEIIeIMRuANhuPxheALh2G13IAy3PwSlUoFMiw4ZFi0yLTqkmLTt9onUqBTITzEgPyXadUKWBerdgeYg6kel3dfp/rsqhYQMqy4WNLOsOujUrMnsreLWx7O8vByNjY0oLy9HJBLBrl27AADDhg2DycTRYUREFB86tRJTBiVhcr4NpfUeHKx2weELRcNTMNwrlyxUSBLSm8ObQDTweYMR+IIReIJhBEJnrhVUSBJSTJpY0My26pFkPPe+qZIkxVayyojTVNoKhYR0iw7pFh3Oy49uc/hCCISbZzM46VydfNpazqFCApKNGqiUnZvInXpO3CaQX7BgAV599dXTtq9fvx6XXHJJh16DE8gTEVF3kmUBd7C51s4fhssfgivQcjsMhy+UkOULW2okW2roMq06qNsJTxFZwBsMw9ccSD3BMIJhGakmLTIsuk6voEPUnTqT1+IWPLsDgycRESWaJxBuNZdogzs6ldO5BNIkgxpZNj2yrdE+h8lGTYebvIl6u16xchEREVFf1NK8nJfcegondyCMRncQ9Z4AGt1BBMIytCoFtGoFdGpl9LZKeWKbSgmtOrqNc0ESRTF4EhERdYBJq4JJq4oNjCGizmOnECIiIiJKCAZPIiIiIkoIBk8iIiIiSggGTyIiIiJKCAZPIiIiIkoIBk8iIiIiSggGTyIiIiJKCAZPIiIiIkoIBk8iIiIiSggGTyIiIiJKCAZPIiIiIkoIBk8iIiIiSggGTyIiIiJKCAZPIiIiIkoIBk8iIiIiSggGTyIiIiJKCAZPIiIiIkoIBk8iIiIiSggGTyIiIiJKCAZPIiIiIkoIBk8iIiIiSggGTyIiIiJKCAZPIiIiIkoIBk8iIiIiSggGTyIiIiJKCAZPIiIiIkqIuAXPsrIy3HLLLRg8eDD0ej2GDh2KBx98EMFgMF6HJCIiIqJeTBWvFz5w4ABkWcYLL7yAYcOGYd++fbjtttvg8Xjw+OOPx+uwRERERNRLSUIIkaiDLVu2DM899xyOHDnSof2dTiesViscDgcsFkucS0dEREREndWZvBa3Gs+2OBwOJCcnn/HxQCCAQCAQu+90OhNRLCIiIiJKgIQNLiouLsbTTz+NO+6444z7LF26FFarNXbJy8tLVPGIiIiIKM46HTwfeOABSJLU7uXAgQOtnnP8+HHMmTMHN9xwA2677bYzvvbixYvhcDhil4qKis7/RERERETUK3W6j2ddXR0aGhra3WfIkCHQaDQAgMrKSlxyySW44IILsHz5cigUHc+67ONJRERE1LvFtY9nWloa0tLSOrTv8ePHcemll2LKlCl45ZVXOhU6iYiIiKh/idvgouPHj+OSSy7BoEGD8Pjjj6Ouri72WGZmZrwOS0RERES9VNyC59q1a1FcXIzi4mLk5ua2eiyBMzgRERERUS8Rt7bvBQsWQAjR5oWIiIiIBh52uiQiIiKihGDwJCIiIqKEYPAkIiIiooRg8CQiIiKihGDwJCIiIqKEYPAkIiIiooRg8CQiIiKihGDwJCIiIqKEYPAkIiIiooRg8CQiIiKihGDwJCIiIqKEYPAkIiIiooRg8CQiIiKihGDwJCIiIqKEYPAkIiIiooRg8CQiIiKihGDwJCIiIqKEYPAkIiIiooRg8CQiIiKihGDwJCIiIqKEYPAkIiIiooRg8CQiIiKihGDwJCIiIqKEYPAkIiIiooRg8CQiIiKihGDwJCIiIqKEYPAkIiIiooSIa/D87ne/i/z8fOh0OmRlZeHmm29GZWVlPA9JRERERL1UXIPnpZdeiv/85z84ePAg3nnnHZSUlOAHP/hBPA9JRERERL2UJIQQiTrY+++/j+uuuw6BQABqtfqs+zudTlitVjgcDlgslgSUkIiIiIg6ozN5TZWgMqGxsRGvv/46ZsyYccbQGQgEEAgEYvcdDgeA6A9ERERERL1PS07rUF2miLP77rtPGAwGAUBccMEFor6+/oz7PvjggwIAL7zwwgsvvPDCCy997FJRUXHWXNjppvYHHngAjz76aLv7FBUVYdSoUQCA+vp6NDY24ujRo3jooYdgtVrxwQcfQJKk0553ao2nLMtobGxESkpKm/v3Rk6nE3l5eaioqGD3gD6A56tv4fnqe3jO+haer76nN5wzIQRcLheys7OhULQ/fKjTwbOurg4NDQ3t7jNkyBBoNJrTth87dgx5eXnYtGkTCgsLO3PYPoP9UvsWnq++heer7+E561t4vvqevnbOOt3HMy0tDWlpaV06mCzLANCqVpOIiIiIBoa4DS7avHkztm7dipkzZyIpKQklJSX43e9+h6FDh/bb2k4iIiIiOrO4zeNpMBiwcuVKXH755Rg5ciRuueUWTJgwAZ9//jm0Wm28DtvjtFotHnzwwX79M/YnPF99C89X38Nz1rfwfPU9fe2cJXQeTyIiIiIauLhWOxERERElBIMnERERESUEgycRERERJQSDJxERERElBINnFzz77LMoKCiATqfD9OnTsWXLlnb3f+uttzBq1CjodDqMHz8eH374YYJKSkDnzteLL76Iiy66CElJSUhKSsIVV1xx1vNL3auzf18tVqxYAUmScN1118W3gHSazp4zu92OhQsXIisrC1qtFiNGjOD/xQTq7Pl68sknMXLkSOj1euTl5eGXv/wl/H5/gko7sH3xxReYO3cusrOzIUkS3n333bM+Z8OGDZg8eTK0Wi2GDRuG5cuXx72cndItC7IPICtWrBAajUb84x//EN9++6247bbbhM1mEzU1NW3u/9VXXwmlUikee+wxsX//fvH//t//E2q1WuzduzfBJR+YOnu+fvSjH4lnn31W7Ny5UxQVFYkFCxYIq9Uqjh07luCSD0ydPV8tSktLRU5OjrjooovEtddem5jCkhCi8+csEAiI888/X3znO98RGzduFKWlpWLDhg1i165dCS75wNTZ8/X6668LrVYrXn/9dVFaWio+/vhjkZWVJX75y18muOQD04cffih++9vfipUrVwoAYtWqVe3uf+TIEWEwGMSiRYvE/v37xdNPPy2USqVYs2ZNYgrcAQyenTRt2jSxcOHC2P1IJCKys7PF0qVL29z/xhtvFNdcc02rbdOnTxd33HFHXMtJUZ09X6cKh8PCbDaLV199NV5FpJN05XyFw2ExY8YM8dJLL4n58+czeCZYZ8/Zc889J4YMGSKCwWCiikgn6ez5WrhwobjssstabVu0aJG48MIL41pOOl1Hgud9990nxo4d22rbTTfdJGbPnh3HknUOm9o7IRgMYvv27bjiiiti2xQKBa644gp8/fXXbT7n66+/brU/AMyePfuM+1P36cr5OpXX60UoFEJycnK8iknNunq+/vjHPyI9PR233HJLIopJJ+nKOXv//fdRWFiIhQsXIiMjA+PGjcOf//xnRCKRRBV7wOrK+ZoxYwa2b98ea44/cuQIPvzwQ3znO99JSJmpc/pC5ojbkpn9UX19PSKRCDIyMlptz8jIwIEDB9p8TnV1dZv7V1dXx62cFNWV83Wq+++/H9nZ2af9IVP368r52rhxI15++WXs2rUrASWkU3XlnB05cgSfffYZ5s2bhw8//BDFxcW48847EQqF8OCDDyai2ANWV87Xj370I9TX12PmzJkQQiAcDuPnP/85/vd//zcRRaZOOlPmcDqd8Pl80Ov1PVSyE1jjSXQGjzzyCFasWIFVq1ZBp9P1dHHoFC6XCzfffDNefPFFpKam9nRxqINkWUZ6ejr+/ve/Y8qUKbjpppvw29/+Fs8//3xPF43asGHDBvz5z3/G3/72N+zYsQMrV67E6tWr8fDDD/d00aiPYo1nJ6SmpkKpVKKmpqbV9pqaGmRmZrb5nMzMzE7tT92nK+erxeOPP45HHnkEn376KSZMmBDPYlKzzp6vkpISlJWVYe7cubFtsiwDAFQqFQ4ePIihQ4fGt9ADXFf+xrKysqBWq6FUKmPbRo8ejerqagSDQWg0mriWeSDryvn63e9+h5tvvhm33norAGD8+PHweDy4/fbb8dvf/hYKBeuvepMzZQ6LxdIrajsB1nh2ikajwZQpU7Bu3brYNlmWsW7dOhQWFrb5nMLCwlb7A8DatWvPuD91n66cLwB47LHH8PDDD2PNmjU4//zzE1FUQufP16hRo7B3717s2rUrdvnud7+LSy+9FLt27UJeXl4iiz8gdeVv7MILL0RxcXHsSwIAHDp0CFlZWQydcdaV8+X1ek8Lly1fGoQQ8SssdUmfyBw9Pbqpr1mxYoXQarVi+fLlYv/+/eL2228XNptNVFdXCyGEuPnmm8UDDzwQ2/+rr74SKpVKPP7446KoqEg8+OCDnE4pgTp7vh555BGh0WjE22+/LaqqqmIXl8vVUz/CgNLZ83UqjmpPvM6es/LycmE2m8Vdd90lDh48KD744AORnp4u/vSnP/XUjzCgdPZ8Pfjgg8JsNot///vf4siRI+KTTz4RQ4cOFTfeeGNP/QgDisvlEjt37hQ7d+4UAMRf//pXsXPnTnH06FEhhBAPPPCAuPnmm2P7t0yn9Jvf/EYUFRWJZ599ltMp9QdPP/20yM/PFxqNRkybNk188803scdmzZol5s+f32r///znP2LEiBFCo9GIsWPHitWrVye4xANbZ87XoEGDBIDTLg8++GDiCz5Adfbv62QMnj2js+ds06ZNYvr06UKr1YohQ4aIJUuWiHA4nOBSD1ydOV+hUEj84Q9/EEOHDhU6nU7k5eWJO++8UzQ1NSW+4APQ+vXr2/xMajlH8+fPF7NmzTrtOZMmTRIajUYMGTJEvPLKKwkvd3skIVhXTkRERETxxz6eRERERJQQDJ5ERERElBAMnkRERESUEAyeRERERJQQDJ5ERERElBAMnkRERESUEAyeRERERJQQDJ5ERERElBAMnkRERESUEAyeRERERJQQDJ5ERERElBAMnkRERESUEP8fD8nButY63PUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 800x300 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Get into evaluation (predictive posterior) mode\n",
    "model.eval()\n",
    "likelihood.eval()\n",
    "\n",
    "# Test points are regularly spaced along [0,1]\n",
    "# Make predictions by feeding model through likelihood\n",
    "with torch.no_grad(), qpytorch.settings.fast_pred_var():\n",
    "    test_x = torch.linspace(0, 1, 51)\n",
    "    test_x_distributional = torch.stack((test_x, (1e-3 * torch.ones_like(test_x)).log()), dim=1)\n",
    "    observed_pred = likelihood(model(test_x_distributional))\n",
    "\n",
    "with torch.no_grad():\n",
    "    # Initialize plot\n",
    "    f, ax = plt.subplots(1, 1, figsize=(8, 3))\n",
    "\n",
    "    # Get upper and lower confidence bounds\n",
    "    lower, upper = observed_pred.confidence_region(rescale=True)\n",
    "    # Plot training data as black stars\n",
    "    ax.errorbar(train_x_mean.numpy(), train_y.numpy(), xerr=train_x_stdv, fmt='k*')\n",
    "    # Plot predictive means as blue line\n",
    "    ax.plot(test_x.numpy(), observed_pred.mean.numpy(), 'b')\n",
    "    # Shade between the lower and upper confidence bounds\n",
    "    ax.fill_between(test_x.numpy(), lower.numpy(), upper.numpy(), alpha=0.5)\n",
    "    ax.set_ylim([-3, 3])\n",
    "    ax.legend(['Observed Data', 'Mean', 'Confidence'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "As a final note, we've made it very easy to extend the distributional kernel class by exposing a generic `DistributionalInputKernel` class that takes as input any distance function over probability distributions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}