{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# QPyTorch Regression Tutorial (GPU)\n",
    "\n",
    "(This notebook is the same as the [simple QEP regression tutorial](../01_Exact_QEPs/Simple_QEP_Regression.ipynb) notebook, but does all computations on a GPU for acceleration.\n",
    "Check out the [multi-GPU tutorial](./Simple_MultiGPU_QEP_Regression.ipynb) if you have large datasets that needs multiple GPUs!) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "In this notebook, we demonstrate many of the design features of QPyTorch using the simplest example, training an RBF kernel q-exponential process on a simple function. We'll be modeling the function\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "  y &= \\sin(2\\pi x) + \\epsilon \\\\ \n",
    "  \\epsilon &\\sim \\mathcal{Q}(0, 0.04) \n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "with 100 training examples, and testing on 51 test examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "import torch\n",
    "import qpytorch\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "%matplotlib inline\n",
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set up training data\n",
    "\n",
    "In the next cell, we set up the training data for this example. We'll be using 100 regularly spaced points on [0,1] which we evaluate the function on and add q-exponential noise to get the training labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Training data is 100 points in [0,1] inclusive regularly spaced\n",
    "train_x = torch.linspace(0, 1, 100)\n",
    "# True function is sin(2*pi*x) with q-exponential noise\n",
    "train_y = torch.sin(train_x * (2 * math.pi)) + qpytorch.distributions.QExponential(torch.tensor([0.0]), torch.tensor([1.0]), torch.tensor(1.0)).sample(train_x.size()) * math.sqrt(0.04)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setting up the model\n",
    "\n",
    "See [the simple QEP regression tutorial](../01_Exact_QEPs/Simple_QEP_Regression.ipynb) for a detailed explanation for all the terms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "POWER = 1.0\n",
    "class ExactQEPModel(qpytorch.models.ExactQEP):\n",
    "    def __init__(self, train_x, train_y, likelihood):\n",
    "        super(ExactQEPModel, self).__init__(train_x, train_y, likelihood)\n",
    "        self.power = torch.tensor(POWER)\n",
    "        self.mean_module = qpytorch.means.ConstantMean()\n",
    "        self.covar_module = qpytorch.kernels.ScaleKernel(qpytorch.kernels.RBFKernel())\n",
    "    \n",
    "    def forward(self, x):\n",
    "        mean_x = self.mean_module(x)\n",
    "        covar_x = self.covar_module(x)\n",
    "        return qpytorch.distributions.MultivariateQExponential(mean_x, covar_x, power=self.power)\n",
    "\n",
    "# initialize likelihood and model\n",
    "likelihood = qpytorch.likelihoods.QExponentialLikelihood(power=torch.tensor(POWER))\n",
    "model = ExactQEPModel(train_x, train_y, likelihood)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using the GPU\n",
    "\n",
    "To do computations on the GPU, we need to put our data and model onto the GPU. (This requires PyTorch with CUDA)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_x = train_x.cuda()\n",
    "train_y = train_y.cuda()\n",
    "model = model.cuda()\n",
    "likelihood = likelihood.cuda()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's it! All the training code is the same as in [the simple QEP regression tutorial](../01_Exact_QEPs/Simple_QEP_Regression.ipynb)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter 1/50 - Loss: 0.944   lengthscale: 0.693   noise: 0.693\n",
      "Iter 2/50 - Loss: 0.913   lengthscale: 0.644   noise: 0.644\n",
      "Iter 3/50 - Loss: 0.879   lengthscale: 0.598   noise: 0.598\n",
      "Iter 4/50 - Loss: 0.841   lengthscale: 0.555   noise: 0.554\n",
      "Iter 5/50 - Loss: 0.798   lengthscale: 0.514   noise: 0.513\n",
      "Iter 6/50 - Loss: 0.750   lengthscale: 0.475   noise: 0.474\n",
      "Iter 7/50 - Loss: 0.698   lengthscale: 0.439   noise: 0.437\n",
      "Iter 8/50 - Loss: 0.645   lengthscale: 0.405   noise: 0.402\n",
      "Iter 9/50 - Loss: 0.595   lengthscale: 0.372   noise: 0.369\n",
      "Iter 10/50 - Loss: 0.548   lengthscale: 0.342   noise: 0.339\n",
      "Iter 11/50 - Loss: 0.507   lengthscale: 0.315   noise: 0.310\n",
      "Iter 12/50 - Loss: 0.469   lengthscale: 0.292   noise: 0.284\n",
      "Iter 13/50 - Loss: 0.432   lengthscale: 0.272   noise: 0.259\n",
      "Iter 14/50 - Loss: 0.398   lengthscale: 0.255   noise: 0.236\n",
      "Iter 15/50 - Loss: 0.363   lengthscale: 0.241   noise: 0.215\n",
      "Iter 16/50 - Loss: 0.329   lengthscale: 0.230   noise: 0.196\n",
      "Iter 17/50 - Loss: 0.296   lengthscale: 0.222   noise: 0.178\n",
      "Iter 18/50 - Loss: 0.263   lengthscale: 0.215   noise: 0.162\n",
      "Iter 19/50 - Loss: 0.230   lengthscale: 0.210   noise: 0.147\n",
      "Iter 20/50 - Loss: 0.198   lengthscale: 0.207   noise: 0.134\n",
      "Iter 21/50 - Loss: 0.167   lengthscale: 0.205   noise: 0.122\n",
      "Iter 22/50 - Loss: 0.136   lengthscale: 0.205   noise: 0.110\n",
      "Iter 23/50 - Loss: 0.107   lengthscale: 0.206   noise: 0.100\n",
      "Iter 24/50 - Loss: 0.079   lengthscale: 0.208   noise: 0.091\n",
      "Iter 25/50 - Loss: 0.053   lengthscale: 0.211   noise: 0.083\n",
      "Iter 26/50 - Loss: 0.028   lengthscale: 0.215   noise: 0.076\n",
      "Iter 27/50 - Loss: 0.006   lengthscale: 0.220   noise: 0.069\n",
      "Iter 28/50 - Loss: -0.013   lengthscale: 0.225   noise: 0.063\n",
      "Iter 29/50 - Loss: -0.029   lengthscale: 0.231   noise: 0.058\n",
      "Iter 30/50 - Loss: -0.043   lengthscale: 0.237   noise: 0.053\n",
      "Iter 31/50 - Loss: -0.053   lengthscale: 0.243   noise: 0.049\n",
      "Iter 32/50 - Loss: -0.060   lengthscale: 0.249   noise: 0.045\n",
      "Iter 33/50 - Loss: -0.065   lengthscale: 0.254   noise: 0.042\n",
      "Iter 34/50 - Loss: -0.066   lengthscale: 0.259   noise: 0.039\n",
      "Iter 35/50 - Loss: -0.066   lengthscale: 0.262   noise: 0.037\n",
      "Iter 36/50 - Loss: -0.063   lengthscale: 0.265   noise: 0.035\n",
      "Iter 37/50 - Loss: -0.060   lengthscale: 0.266   noise: 0.033\n",
      "Iter 38/50 - Loss: -0.056   lengthscale: 0.266   noise: 0.032\n",
      "Iter 39/50 - Loss: -0.052   lengthscale: 0.265   noise: 0.031\n",
      "Iter 40/50 - Loss: -0.049   lengthscale: 0.262   noise: 0.030\n",
      "Iter 41/50 - Loss: -0.047   lengthscale: 0.259   noise: 0.029\n",
      "Iter 42/50 - Loss: -0.046   lengthscale: 0.254   noise: 0.029\n",
      "Iter 43/50 - Loss: -0.046   lengthscale: 0.249   noise: 0.029\n",
      "Iter 44/50 - Loss: -0.047   lengthscale: 0.243   noise: 0.029\n",
      "Iter 45/50 - Loss: -0.049   lengthscale: 0.237   noise: 0.029\n",
      "Iter 46/50 - Loss: -0.051   lengthscale: 0.231   noise: 0.029\n",
      "Iter 47/50 - Loss: -0.054   lengthscale: 0.225   noise: 0.030\n",
      "Iter 48/50 - Loss: -0.057   lengthscale: 0.219   noise: 0.030\n",
      "Iter 49/50 - Loss: -0.059   lengthscale: 0.214   noise: 0.031\n",
      "Iter 50/50 - Loss: -0.061   lengthscale: 0.210   noise: 0.032\n"
     ]
    }
   ],
   "source": [
    "# Find optimal model hyperparameters\n",
    "model.train()\n",
    "likelihood.train()\n",
    "\n",
    "# Use the adam optimizer\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=0.1)  # Includes QExponentialLikelihood parameters\n",
    "\n",
    "# \"Loss\" for QEPs - the marginal log likelihood\n",
    "mll = qpytorch.mlls.ExactMarginalLogLikelihood(likelihood, model)\n",
    "\n",
    "training_iter = 50\n",
    "for i in range(training_iter):\n",
    "    # Zero gradients from previous iteration\n",
    "    optimizer.zero_grad()\n",
    "    # Output from model\n",
    "    output = model(train_x)\n",
    "    # Calc loss and backprop gradients\n",
    "    loss = -mll(output, train_y)\n",
    "    loss.backward()\n",
    "    print('Iter %d/%d - Loss: %.3f   lengthscale: %.3f   noise: %.3f' % (\n",
    "        i + 1, training_iter, loss.item(),\n",
    "        model.covar_module.base_kernel.lengthscale.item(),\n",
    "        model.likelihood.noise.item()\n",
    "    ))\n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Make predictions with the model\n",
    "\n",
    "First, we need to make some test data, and then throw it onto the GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_x = torch.linspace(0, 1, 51).cuda()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now the rest of the code follows [the simple QEP regression tutorial](../01_Exact_QEPs/Simple_QEP_Regression.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get into evaluation (predictive posterior) mode\n",
    "model.eval()\n",
    "likelihood.eval()\n",
    "\n",
    "# Test points are regularly spaced along [0,1]\n",
    "# Make predictions by feeding model through likelihood\n",
    "with torch.no_grad(), qpytorch.settings.fast_pred_var():\n",
    "    observed_pred = likelihood(model(test_x))\n",
    "    mean = observed_pred.mean\n",
    "    lower, upper = observed_pred.confidence_region(rescale=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For plotting, we're going to grab the data from the GPU and put it back on the CPU.\n",
    "We can accomplish this with the `.cpu()` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean = mean.cpu()\n",
    "lower = lower.cpu()\n",
    "upper = upper.cpu()\n",
    "\n",
    "train_x = train_x.cpu()\n",
    "train_y = train_y.cpu()\n",
    "test_x = test_x.cpu()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQIAAADDCAYAAABpjB/1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2dd3hT5f7APxltulsK3S0tQ4Ys0YNURGWJCCqKIugVUK6oV3FeRlGGDAUB5QKKyE+v4woKCAoICIoDkIIehqwyympL995JmuT3R5rQtGkbaGgDvp/n6fM0Z7z5ntOe73nf71SYTCYEAsHfG2VTCyAQCJoeoQgEAoFQBAKBQCgCgUCAUAQCgQChCAQCAaBu6ACSJCmBTcA+wB1oA4yVZbmsoWMLBILGwVkzgnhZlmfJsjwV8AKGOWlcgUDQCDR4RiDLshGYAyBJkhqIBE42dFyBQNB4NFgRWJAk6R7gVeB7WZbl2o6Li4sToYwCQRMxb948hb3tTlMEsixvA7ZJkvSFJEnPy7K8rLZjZ86cWe94mZmZBAcHO0u8q4Kry+jq8oGQ0Rk4Kt+MGTNq3ddgG4EkSTdKkjSkyqZzQOuGjisQCBoPZ8wItMA/JUnqDrgBHYGXnDCuQCBoJJxhLDyD8BII6qGiooL09HS0Wi1VM16NRiOFhYVNKFn9uLqM1eVTKBRoNBpCQ0NRqx17xJ1mIxAI6iI9PR1vb28iIiJQKC7Zq/R6PW5ubk0oWf24uozV5TOZTOTn55Oenk5kZKRDY4jIQkGjoNVqCQgIsFECgquDQqEgICAArVbr8DlCEQgaBZPJJJRAI6JQKLicokNCEQhcjrS0NAYMGEB6evoVj7F//34mTpzI7Nmzee211zhz5gwA8+fP5+mnn3aWqA5TUlLCyJEj+eKLL2y2//LLL7Rr146JEycyY8YMXn/9dUpLS2sdZ+PGjeTn5ztdPqEIBC7H3Llz2bNnD2+//fYVnV9YWMicOXOYN28e06ZNY/bs2bzwwgsYDAYeffRRJ0vrGN7e3gwePLjG9r59+xIdHc0LL7zAzJkz6d27N5MmTap1nKulCISxUOAyBAQEUF5ebv28YsUKVqxYgYeHx2X982/dupXY2FhUKhVgfghbtWrFH3/8QVhYGMnJycybN4/9+/czbdo02rRpw0svvUTHjh05c+YMH374Id999x3x8fEYDAaGDBmCt7c3Tz75JCNGjCA+Pp7Ro0cza9Ys1q1bR1JSEkuWLGHdunVMnTqV8PBwkpKSeOuttzAajTz//PO0b9+e5ORk7rjjjjplHzx4MJMnT8ZoNDJ//nz0ej1JSUlMmDABtVrN4cOHef/997n33nsxGAxs2bIFpVJJ7969GTbsyp13QhEIXIaEhATi4uLYuHEjZWVleHp6MnToUObNm3dZ46SkpNC8eXObbS1atODixYuEhYUBEBcXR2JiIpMmTWL27NkUFRXx8ssvc+jQIQwGA3PmzOHPP/9Ep9MxaNAgfvnlF3r16sVdd93FK6+8QnFxMeHh4cTExJCamsqbb77Jjz/+iLe3NxMmTGDVqlX873//Q6vV0r9/f8aOHcvs2bMdkr9Zs2ZkZ2cTGxtLnz59OHz4MB9//DELFiyga9eujB8/npiYGPbt28e8efNQKpUMHTpUKALB9UFYWBh+fn5otVo8PDzQarX4+fkRGhp6WeNERkZy/vx5m21ZWVlERUVZ9wO0bNmSM2fO0KlTJ2JjY+nbty+DBw+mZcuWFBYWsnDhQgCb8N2YmBj8/f3x9/fngQceYNOmTSQmJjJ16lQWLlzI6dOnWbBgAfn5+URERHDmzBkefPBBAOv310d+fj4tWrSgsLCQ6dOnYzAYyMvLq3Gct7c3s2bNws/Pj9zc3Mu6R9URNgKBS5GZmcm4cePYuXMn48aNIyMj47LHGDRokHVaD2ZDXXJyMpIkAXDx4kUAkpKSaN26NefOnWPYsGHs3LmTHTt2oFAoCAkJYeLEiUycOJGRI0fa/Z5HH32UL774Ak9PTwDat29P165dmThxIq+88go333wzbdq04cKFC4B5plIf27Zto0+fPiiVSiZOnMibb77J2LFjrftVKhUmk4kTJ04wY8YMRo4cyb///W+8vLwu+z5VRcwIBC7F6tWrrb8vXrz4isbw9/dn2rRpxMXF4efnR0FBAUuWLEGlUrFmzRrKysqYPXs2f/31FzNmzECn0zF37lxatWpF165dCQ4OZtKkScTFxeHm5ma1HRw9epQVK1ZYjZjh4eEYjUaGDDGn2gwZMoTff/+duXPnkpaWxuTJk+nQoQP/+te/SE5O5vTp0xw9epSHH34Yb29vAHbu3ElSUhLLly/H29ubsrIy61Jo0KBBvPTSSwQEBHD06FHOnDlDr169WLBgAR07duShhx5i5syZxMbGcvHiRX7++Wf69et3RfdM0dgNTuLi4kwi+7BxcCX5EhMTadu2bY3trh61B64vY23yVb/nM2bMqDUNWSwNBAKBUAQCgUAoAoFAgFAEAoEAoQgEAgFCEQiuQ7Zv3067du346KOPbLYPHDiQ4cOHNyiZ6XpFKALBdcfAgQMZOHAgH3/8sTUV9/jx4+Tm5nL//fdfdqTi3wERUCRodDw8NFU+aWo9rj7Ky2svvBESEkLHjh3ZsWMHAwYMYNOmTdx3330A/P7776xduxYfHx+6devG8OHDmTBhAkFBQZw6dYq5c+dy/vx5nnzySUaPHs2BAwe47bbbePXVV69YVlfHGS3P2mBucHIAc3OTHFmWZzV0XIGgoTz//PPMnz8fSZLw8/NDp9MBMGnSJLZt24aPjw933nknjzzyCPfddx99+vRh06ZNrFmzhvHjx9OrVy9iY2N57bXXGDhwoFAE9RAIfC3L8gYASZKOS5K0WZbl/U4YW3AdUvVNfjWj9mJjY8nOzmbevHlMnDiRZcvMrTZSUlL48MMPAXMCUmFhIWfOnGH37t2kpqZaMxQBoqOjUSgULh1Z6AycUcX4z2qblEBJQ8cVCJzBc889x65du2zSklu3bs1LL72ERqPh+++/p6KighUrVrBv3z527NjBnj17rMf+XcqrOdVGIEnSQ8A2WZZP1HVcZmZmvWNdjSoszsbVZXQl+YxGI3q9vsZ2S4agM9m9eze7du1ix44dDB8+nOHDh3Pq1Cl27drFkSNHmDlzJpMmTSIgIICgoCAGDhxITEwM//73vykpKeHs2bPs3buXI0eOsG7dOrp06UJSUhI//vgjffr0cbq8DaW2e2g0Gh161sC5vQ/7An2BV+o71tFEGFdJmKkLV5fRVeQrLCysdXrt7Gl337596du3r822du3asX37duvnO++802Z/1axHC/v27QPMy5eTJ127r6+9e6hUKh3++zvFfVjZ8uwe4GUgVJKk25wxrkAgaByc0fvwFmA1EAv8AmwA2jd0XIFA0Hg4w1i4H/BxgiwCgaCJEJGFAoFAKAKBQCAUgUAgQOQaCJqIpb+YW5AZDEZUqst/H73Yt029xxw4cIDVq1fj4+NDYmIivXv3Zty4cQ6Nn5yczBtvvIGvry8tW7akZcuWPPbYY9b9q1ev5uOPP+bHH3+8bNldEaEIBNclBQUFzJw5k/Xr16NSqSguLubZZ591WBHs2rWLfv36MXr0aPR6PRqNbXLUiBEj+Pjjj6+G6E2CUASC65ItW7Zw++23W9ue+fj48P7777N8+XKysrLIysri+eef59y5czz33HPExcWxZcsWRo8ezb333su3334LQGhoKOvXrycqKopp06Yxf/58Ll68SMeOHa3fdeLECZYuXUp4eDj+/v6MGjWKBx98kI4dO6JUKsnLy2PlypWUlpYyZcoUoqOjkWWZZcuWcezYsRqZkE2BUASC65LU1FQCAwNttun1ejZs2MDWrVs5e/YskydPZu3atbRr145BgwYxYsQIxo4dy/Dhw7n//vsBc28Bd3d3fv/9dzIyMti+fTs//fQT58+fZ926dQBMnTqVuXPncsMNNzB06FAee+wxnnrqKc6fP8/06dN5+OGHycrK4uuvv+amm27iqaee4pdffsFkMtnNhGyK/AahCATXJZGRkdYOQxbOnz9vDbmNiori1KlT1n0xMTEoFApKSmrPlzt//jwRERGAuV2ahdOnT7N+/XqUSiWBgYHW9mPR0dGAuZdhSUkJp06dYujQoQDWEGh7mZD+/v4NuvYrQSgCwXXJ4MGDGTt2LEajEaVSSU5ODnPmzLHWJEhOTqZdu3bW4x15C8fExFjbpSUnJ1u3t2/fnscff5yoqCh27NhBeHi43THbt29vbXv2888/06VLlxqZkH5+fg278CtEKAJBk2Cx+l+tegS+vr5MmTKFKVOm4OPjQ1paGkuWLOGHH35g+vTp5OTkMHv2bPbv309SUhLffvstSqWSpKQkdu7cyZYtWwDo2bMna9eu5ezZszz99NMMGDCAl156icjISJKSkoiPj2fOnDksWLCAoKAglEold955p835R48e5bvvvuOZZ54hLi6OzMxMSktL6devH2+//TZxcXEEBAQQHBzcZGnPouVZA3B1GV1JPtHy7OohWp5do6SlpTFgwABRTVfgMghFcBUwGk0UlunJKdbZ/BSVV2A0mpg7dy579uyxdtX9O6BQKGjs2effGZPJdFnLDGEjaCD5pXrSCspJLSgnt0RHsbaCEp3B7j/9lAe6U6HXWT+vWLGCFStWoNFoKCgoaEyxGx2NRkN+fj4BAQF/m/JfTYXJZCI/P79GEFRdCEVwmRiMJpJySzmdWUJCchZK9yKHz53y2Xa+/78FHI3fgV5bjpvGg869+nPf0xP55PcLRDXzpE2QN9GBnqivIOzWlQkNDSU9PZ3c3FwbJWmx6rsyri5jdfkUCgUajeay+jcIReAgF/PLSEgv4mxWKdoKc424Mr0Bb3fb4wpzs/hy7gSemLIQv8Agm31+gUFovLyp0GlRu2uo0Gnx8PLBLzCIUl0FJzOKOJlRhJtKSXRzLzqE+BDT3Ou6eIOq1WoiIyNrbHclg2ZtuLqMzpBPKII6MBhNnM4s5lByAVnFtTfTqMpPqz7k/LH9/LTqQ4aNn15jf3F+LrGDRxA7eDh7t6ylKC+7xjF6g5HEzGISM4vx9XCjc7gvN4b54eWuavA1CQT2EIrADnqDkb9SCjmcUkCJrsKhc6qv/+M3ryZ+82rUbu7M3XjQun3MtMXW34eNn1bvuEXleuLP5rLvXB7tQ324pWUAzbzc6z1PILgcXHfh0wQYjCYOpxTwxd5ktsknWPDyPyjMzarznMLcLJZNHMP4RV/Rvc8Q3DQeALhpPOjedwhTPtte5/mOjF2Ym4XRZCIhrYiVf6Sw/XgmuSW6+gcQCBxEKALMVtYT6UX8b18yv53OplRXYZ3ib/nve9aHEWwfTri0FNi3dY3d9T9gc3xd1Db2T6s+tO7/YMJo/kw4y6pKhVBYVrNXgEBwuThlaSBJUijm/ofdZFnu4YwxG4uMQi07T2eTXlgO1Jzi79+xEYA5T/Rj/pYj1odzzqh+mIxG63Hxm8118RUKJS8uWmVd/1d9mAc8/i+7hkSLgTEwJLzWseM3r0ahUAImq/3hZEYRiVnFdI3wp0d0ABo3YUMQXBnOshH0xlzG/CYnjXfVKdMZiD+Xy/HUIkxccmdZXHwHf91sc7zJZGTivZ2qbrDZX9UV6BcYxB/bNmCosFj7PYnfvIn4zWsBo/VBtiiA88cPYDIaOXfU/thVZYCa9oeDyfkcTyuiZ6tmdAn3Q6m89r0MgsbFKYpAluVvJEnq4+jxTdnyLCMjg7HPPs/9L87G3Tewxn6VxguluzkQo2o0nMrNHZ+A5pQU5FKh06JQKivf2j4oVXeg1/bkQkI/Ppt9EwXZ7hgNJUD1B1IPpBC/OYn4zfuBk4Av4En1dpGBoVHkZqSgVrtRodcRGBpFYW6mdenRsWdfBo5+yZo2WwJsPVTIvlNu3BbtS4ivu0u1PKsNIWPDcYZ8TeI1aKqWZ3mlOqa8MoWjB2V8vvnUrnsPQFtcyG1DRlJckMuR3dtRKJQYK/R4eHpRkJUG3IrJOBxzh7ebMBrMU/LcdPMPgEJhQqkqx2jtS6cAvIBWlT9VMQCHgd9A8R2YdmMyGbmtipvxhLwLg15ntT+cP7YfLy8vvL29bWUHfkvS0T7UnXZ+fi7t/7YgZGw4Io7AAYxGEwHNAtBpL8UC1Obeg0suvs9nv8xtQ0YSO3g4O77ey4k/I3D3GImuPKTK0Xq8fBPp0ENJ0smV3Pvkg0S08UapSuM/44fSsedd3PnQaFa9M4mMpIuo3Fpj0IfiE3AzpUXRGA23At0v/ZhewU2Tj4KddLuzK2GtShk2fhqfz36ZDtIdxA4ezqp3JpORlFhrrIIJs/Hz6IUy7lF6cWNY0+S4C64drntFkFWkZceJLCb/d1ut4b21Mer1xRz/w4cNHzbn7NFh1u1umlz02s9Rqn7AaPidbnfeB0BO6hoSD52ga+/prH9/GWXFBbhrPAhv3YGgyFa07tKjyhv+K0wFqZXLDw2tuzxPecldZF2U0GvDyM14gOWTIThKy21D8nj01aXMerwb8Zu/tspRlzID0FYY2XEii5MZxfRtF0SAl+um0gqaFmd5De4CRgFhkiRNBd6VZbnMGWNfKQajiT/O53EgKR+jyVRneG919DoFf/wQwK7vmpOTZg7e0Xga6N63gJvuKmTXd8/gF9icbn1H89Gkn6weA7j0cFb/XPVhrf6GN3sY/uJf88diMuVx8UwZR3b7If/kT2ayhg3LQ9nyaTBd7zhHecl0Th9a6bAyA0jJK+OrP1O4tVUzukf6C2OioAbOMhb+BvzmjLGcQXpBOTtOZtUIurEX3ls1N8DbPwh5ewA/rgqiINv89mwWouOOobn0GJiPh7fZat+m638AKCkpYer/fraZaajdNfgGNKcoP4cKnbbWh7W2CEOFAvwCkzl3bALj33uX5FOt2PWdD+ePNePAz+Gg+AhMd6ByW0iF7lityqw6FUYje87kcDarhP4dggisniQh+FtzXS0N9AYje8/l8VdygY1L0IK9h2/9+7M4d3Q/Xy88RF7mc2RfNHsMwlqXM2BkFp17FaGswz1ffaZh0Otw9/C0Mew5+rBasMQe/LJmGcPGTyfx0ETOHztMUOQyslLuAMZgrBhNYNgestM+cnhcgPTCcr6WU4htFUj3KP/rIqFJ0HCuG0WQnFfGLyezSL6YWmv2X1UuBQ51B3Zz+mCvyj2n+cdkT7reWYijmafVZxpH43fUm1hUt0xmqi8zslL6Aq1QKN5AqRpLTurt5Gf2Yu3ii6Sff5Ix06bhFxhkneUMe3lWDa8CQF52Jv/492hee/t9hvfuJGwHgms/xLhUZ2D78Uy+O5RKQZm+Rlhubby89Geah20BZKAXkA6Mo0uvl9mz+SGK8+sPCbYwZtpiho2fRnjrDgwbP43pK3+1+Vx1JlIXUz7bbpOvoHbX0Cw4HHVlXIM5f+FGpn7ZlsmfnEa6Ox+jAf74IZKkE9/w+exkKnQK6z34be0ndr/Hsn/VR4v4Wk7haGqhw9cquD65ZmcEJpOJo6lFxJ/NRVthcDj7z2iEP7cHsPmT9pQVqzAH+bwHzAKKOLLHfJwlpLgxubxlRgWHfg3BZOoAzAfuIenEE0wZegrIBUzI29chb19nvQd13aOv95yiX/sgker8N+WanBGkFZTzzYFUfj2VZS0SUv1tai/7LzPZnY8mR/PN4nDKilV4++/nxthXaS9tBGwrDVlCiqc80L3RrgsuLTNeXLSK2MEjKC0utPlclJdjPXb8oq/w9k9B7T4UGAicANoB24G1qNza2tyDuu7RuewSVv2RzPmc0ka9XoFr4NIzgrS0NEaNGsWXX35JaGgoeaU64s/mciarZjea2tyDAB/8+5+EtvqEvVtagskdn4AKhj6bTre7PFEonmfdUnN5dYVCaY3nd9Q1B+CjURPm70Ggtzu+GjU+Hmp8NWrcVLaGOJ3BSHF5BUXaCorKK8gp0ZNeUG5T86C6QbOqR6F6/YJ9W9dQUmDuqqN230mFrhs+ATMpzh8PPIJBfy+56esxGswZkE9MWVhnhuQTUxaySW+ga6Q/t7cOvO7KpQlqx6UVgaXa75uzZvPYKzM5mlqIsY5KuPbcg98t28b54ys4f/xGAIIif2H8e2F4+RptzqseUlyXtd9dpaR1kDfeJjWdW4Xj5+mYsc0b7BYVKSzTk1aoJTmvlHPZpZTrDTVPrkL1KT5AhU4LKCgtnkaX3qcpL5nJ6YORXEgYxX9ePENJgTm12t49ql5V6XBKAcdOJ7FmwUS+/mrlZdW+E1ybuGSDk4CAAMrLy2tsry2Czu733H8bhorXgQmACkgExgG/1jrW57NfxrdZC5uHxPKGVikVxDT3pl2wNzHNvVCrlFellp3RaOJifhmJWSUkZpXYVQqFuVl2oySVShUHft5E7OBHGTZ+OpPvi8NoeB9oCVQA7wJvonYz2rUZWFC7udNj4EPs3bKGYY+PYeUny516jVVx9XqA4PoyOirfNdfgJCEhgdv6D7axnnv7B/Lif75y6PxzxzzxD0oDJlduWQh0BX5F7a6he98hjF/0VY2CIdWt/2OmLcbLXU3PVoHc28qd9175Bz7G4qs6ZVYqFUQFetG3fRBP3daSuzsGE+rnYXNM9WWQXlvOwV82s3/HRkwmE/GbVzPx3k4YDd8DnYAlmP/Uk9F4nWLUG/GAfZuBQqmkQq8jfvNqTCYT61Z+hoeHB/7+AVftmgVNj0sqgrCwMFQeXjZr2ZKCXPZuWVPnebpyBRuWh/DhxBhy07zw9EkBeqNQTgbKUCiVGPQ6PLx82Ld1TZ1uxkAvd/p3CGJMbBS3xjRj0cL5jd6URK1S0iHUl+G3RDBCiqRDqK81AKiqUfGW/kPxbxFifaAvYQKKgZcxu0iPoy2N4rNZ3dn8STCePsE1bAY3973PrkFx9qodZBTWnKUJmpaL+WWU6upeSjqCSy4NAHr2G8zhvb/aVOqxYG9an3jIi7WLw8lNd0epNNH30WzSzo/Bv7k/mSnnKM7PwSegOWeP/FnnmL4ebsS2akb7EB8UCkWtyxQPDw9OnTrV6FPG/FI9+5PyOZFeZGMvWbd0Jvu2rkXl5k6FTkuL8JbkZ2dU1k5Q0e6WXnj5hJB86mFy0kZgMioIjtLi2+w1gqNybZZDPgGB1rEMep11qaFUKOjVJpDuUc6bHbj6tBtcV8YzWSVsO57BPa08aNMyvN7j61oauKyx8Jnp75KeU1BnxmBhbhafz3mTFmErzXH4QEjLYh6bmElE23LMSwJbaltfD/9XHHfe0ILO4X6oqiTlJCQkEBcXx8aNGykrK8PT05OhQ4cyb968xrgNNQjwcqN/hyB6RAfw54V8EtKLMJlMNYyAVesXGPQ6AoPDrSnLFxLOs2ZROJnJGrJT36dN1yxCorMZNn4ahblZvPvcg9zc/wHufGi0TWSk0WRid2IOqfnlDOgQJEqjNSFHLhby26lsu6H0V4LLKgKo2yW4bOIYFIphJCWsJCkhHIWiApNpFjGdzhDR9o1aG43YGzMyuDnjB9+Cu7rmSiksLAw/Pz+0Wi0eHh5otVr8/PwIDQ11qNLS1cLP06wQukf58/uZ3Bpux89nv0ybm2K544HHa4Q5R3cs45WlZ9n6eTC7vm3O9i+DOfGnDyMnprLr2w9t0qftlVw/m13C17KOQZ1CCPFzvK2W4Mqp6kq/UObOH+fznDq+SysCsO8SnP3EE2BaDFhqBOzBZHoaSGDfVti3dVWNQp/2xhw6chQJv3xLfk6WXSVgITMzk3HjxvHPf/6TTz75xKW6GAd6u3N/11CS88r4PTHH2ohlzLTFlJSU4O3tbfdhdtOY6PPIUU4fXE1p4XsknfRi/tNhgB9gqrfWQWG5nm8OXOSOG5rTNcL/Kl+lwOJKf2HSNPo8Fef08V3WRvDJrwmUmmx97nH398BQ8RzwJuZaf0WYPQPLUSgVdtf+Fqr+Q3u5q7mjbXPahfhc6WUArrd2NJlMHEktZO/ZPLQVBqsiqI31789i75Y1SHc/RWnR2xyLt1ReWo/a/UW63N7dWoy1rlZu7UJ86NsuqE5lWhuudg/t0ZQyOuJKH9zGs8E2Apf0Gtjj7BEvAoJSMfvCfYFvgRtRKD8CTJiMRmtyTovwlnbDaBUo6BTuxxO3RjZYCbgiCoWCrhH+PNEzio5hvihqFE81M+WB7ky8t5PVRfjn9v9yLD4Uc22ZQmAYFbp9aEtvtT70dSVzncooZu3+i6LpylUgISGB4Y8+irsTG+fYw+WXBrkZbmz9LJhDv1qmn2eAF1EotmEyGely+0CSTx1DAYyZvsRqKKtuV4iOjKBv+xZEBHg24dU0Dl7uKgZ0CCbETcfhbBO5pbYPaG1dmUuLLuDlO43089NJOxfJ8X1TmTRkASbjG5gLrNaezJVbqmPN/ov07xDEDcHXn5JtKvwCg8jWqtBXiRlJ/OsPp3+Py84ISotVbP4kmAXj2lQqgXLMS4LOwFZMJqO5urDByOufbWfKZ9utxq2INh2tPvbbhoxArS1kpBTxt1ACVQnxdWdkj0hujWmGUqGwdlIC7Bphn569nMcnPcvLSzMY8HhW5XJrMt5+h1G73wDU/UbSG4z8cCyDXYk5GI2Nu+S8Hskv1fPNgVSysrKs/88hLdtSlJtl0/1qzD9GNthu5XIzgooKWLFCyZtvdqSk0Cxe59szMBniOHXw6xpuRHu5ABYLegsfDa8+OoBg37+vZVulVNCzVSA3BPvwxNNvWaf3dXVlVqngnlFZtO1Wwqr5ERTm3AjsQ6l+lgrdN/VWXDqUnE9WkZZ7bgzGW+Ny/2LXBJlFWjYdTqdUV8GYaYuZ8kB3u4VrzYlynrz99tssWbLkir/PWcVLB2A24WcCJlmW67cG1oJCAZ9+qqKkUEnrziXcNy6DqHblrFta6lDhUQClQkGPmGbc0jLAJibg70p1g5Ol6pHazb1GhmNV2nQt5bUPzvLuv/IoyovFWLGG0OhtFOQsrtN4COaIt9XyRe7tHEKYf/WIR0FdJOeVseVIOjrDJeN39eWcGTdMpveA3qxY0YsVK1bg4eFxRQ1PGpGNpAEAABlcSURBVLw0kCTJC1gOvCrL8ptAV0mS+l/peCoVLFpUwTMzzvLc/AtEtTNfdPU8/ap5+VUJ8tEwQorg1phmQglUkpCQwIgRI/D0NC+NPD096TVwqEMGJ4MhnebhLzLwiURUahPpF+4h8a+FfPvBl/VWgirRVbD+YCqHUwqcdi3XO4mZxWw6nGajBKBm/AuEonb/HRgPdMTd/S5GjhzJiRMnruh7nTEjuA24IMuypXvI78AQYEdtJ9QXiNOuHdxQmEFJ6SXxHnntUoz/PU+9BmBt9wXmWcBNEd50DXPDWFpAZiPU17hWWmGpVCrc3NwoLy9Ho9FQXl5O62BferYN4UBKcZ2p3Vs/X8qF4/tpERHHU7Nn8fEUL3RlN3J0zwIgy8Z4OPWr3RTlZfPNojd45NW38G3WAoAth4o5mezJ7TF+qKvVaHD1ewiNJ2NCRil7LxTVGi2Yn53JLQOHEXnDODb/Xw/02kAgBbV6JHr9HtTqf6BUKq8o0M0ZiiAY2/I+hZXbaj/BAZ+sl1cOmBwruR3sq2FAh2Ca+zR+iW5X94Fb5CsqKuKZZ56xCYoa0K0VN7XRsv14FjklWpvzqqcoW8qeQQDwKfAg8B3wHl5+7/Ds3A/x9vZm26fvkpRwiD3ffW4TyHUqNYvZr4zlm6+/ol2rSLsyujJXW8b4s7kczjbg5e1V6zH/nPkB8VsC2PBhGIYKBV5+h+l46zJef/F1tm/ZSHp6+hXL6QxFkInZsW/Br3LbVUelVHBrTDNujgoQTTvqYfXqS9WQFy++FI7cwkfDo7eEs/dcHoeqlIGvvia17dfwEPAq8A7wGqWFt7Jo/GNgSrGOa5kpqNRutOzQjcCQcE4flvnXpKn834cf0LpF7YFOfyeMRhO/nMrmeFrdBWSNBtj0fyHs3tAcgJ6DUshIfobBT71Dh44tGXLPFa/GAee4D+OBaEmSLKb524HNdRzvFEJ8PRgpRSJFNxNKoIGoVUp6t23O0JvC8Km08tdVSNUcvr0IuAswp3pjkjE3hTVjcTMajQbOHZWttRJ+3/Q1N0Y2x8/f/2/vYtRVGNl8NKNeJVBequSzWVHs3tAcldrEiNcuolRN4MLxP+qt1u0oDVYEsiyXAv8ClkiSNAc4LMtyrfaBhqJSKujVpjmP3BwuuvU4mahmnjzWI5I2QeaAoNoKqb7y/trKGghnUbv3An4CQoAfgddRuXlYi6XYC/vu3ncIk/+7jY2H0yjT1x4Wfj1TqjPw7aFUzufY1t+0xHpYCubkZ6lZNjGGhD988fKtANMAVr8XaY0Kjd+8mk7tWhMQ0LDUcGe1PPsR83/BVSXUz0O067rKeLipGNw5hGOpnqhnLKWi8kGu7mYcOeHtKjUQHsCgnwJMA94iusML5GcPIjftCEqlCqPRHJWoUCrBZLK6fpPzykjKKOdhb/+/VbBXbomOTYfTKSzX19hXNZQ7dvBbfDK9JYU5brSI0DJ2ZjIaz+l8/3+eNlGhAwfewwdLHeudURvXRLSHWqkktnUzbooULboai07hfoT5e7DteCbZxVq7x1QNSjK3ao8HVnL2SDiwHhiG0XipN0SX2wfg7RdoE7xUqjfw7aE0YlsFckvL6//vezG/jM1HMqxl+C3U7DmRRfzmIMCN1p1LGDM9ubLgbs00eh8fnwYXmHXJEOO0tDQWTfgnhblZhPl7MLJHBN2jAq77fxJXI9DbneE3h9OlljTjMdMW8+f29Sx64WEykhKBrcAtwEGgLbAXFKNo3+MObhkwFKPBaLfzk8lkIv5sDt8fyaDMCWW3XIm0tDQGDBhAeno6x1IL+e5QWg0lANXrRz6G+V76cWPPTMa9lVSj6nbVJVtOtmMt9erCJRXB3LlzOXPsIEc3f8rD3cPtlgAXNA5qlZI+7VowuHMoGnXNikQ1C6CmERg2ArOL0QtMX1CQ+TqPvPS23dZvRXnZ1jXx+ZwSvvozheS8sqt8VY2HpY7A+EnT+PlkVq0xG36BQbh7eqPXvgSsAtxx8/iQYeMTULvbnlO9yO7iDxpuMHQpRRAQEICHhwcrVqzAZDLx7arP8fT0bLAhRNBw2gR5M1KKqLeicoVOi8lYSuzg7fQfeRCFUk/6hUEsj4uhIKfmSvS3tZ9w/th+tvz3PZZNHMOphKPcc/fdfL83AcM17FWo+r9sNBr5fs3/6uycZTRC4qGRmF2y4BPwFvry59nxtXO8AvXhUorAXihsQ8ImBc7Fz9ONYd3D6RZpu1SoPlWNaNORh1+cxqAxGsa/l4x/Cz0Xjnvxnxdbc+awOWDGUhNB3r4Ok8nE/h0bOXdU5j/jH+bcsf28u2Ae6w6kkld6bdY4SEhIYOjDw2vUEbBXRr9Cp2DVOxHkpD0EaIERFOdPBbCWpq9NgdwY5oenE/pVupQiqFofUKPR2NQHFLgGKqWCO2+wXSrY6wdhoWX7cl5ZepY23UoozlOzYko0v60LJO5T85LCHha32OjebYgKDeJAUj6NXUmroWQZvcgqV9aoI7Dr289tcjTKSpR8PK0lf+30x8PLwKjXT9C9T0m9PT283dXc3zWM/h2CUDshjsalFAFcqg+4YcMGxo0bR0ZGRlOLJLBDmyBvRkgRBPnUn+LtE2Bg3FsX6PNINkajgu8/DmXj8u6o3c1RcuYAJVssjWimfLqN38/k8M2B1GuiApKuwlyT4bdT2RTm5VhnSqCgKDerWhOau5n+iI4zh73xCdASGDaKmE75NZZa1Xt6tAvx4fFbI4lpXns48uXicu5DSyhsZmYmffr0aVphBHXi7+nGIzeHszMxh2OpdUfHqVQw5J+ZtOxQxup3wzm82w93z9l0ur0ZSkUCR3ZfyoSs2ojGkuKcXljO13IKPaKb0T3K3yUbtCbllvLzyWyKKuMD7NURuERnFIofMJkiCIrUEnnDBA79+jU/rTJSnJ8LCkVlP0szlpBtd42GwgLnZ3O63t0UXFOoVUr6tQ/i7o7BqJX1/zt1ub2Il5ecIzSmHF1ZS07J75KX0ZfbhoykTbeehES3pXWXHnZTzQ1GE3vP5bLyjxS7HbGbCq3ewE8nMtnwV5pVCVio7lUBQDEA2I3JFAHsJislnIO/vG+dKRzd8xNKpapGx6mhDw/n1MmTV+UaXG5GILg26RDqSwsfd7YczaCgrGbEXFWCInW8uOgc3ywJ4+AvAaScjiOiTR7/nJmOm6Z+W0BhuZ4tR9NpGehF7zbNmyTrFMy2jNOZJexOzLFpbV+Vql4VczWhx8H0X8ANWIu5YOylN3/V6ls/rlxGhU6LW2WuR0jzZlfNXiZmBAKn0cJHw4hbImjlQGahu4eJxyamct+z51G7Gdn3QzOWvtaKrIt1P9RVY/GTckv56s8UthzNIKvIfvTj1eJCTimr5YtsO55RqxKwUJyfS897R9LjnmPA/wA3lKrFwAjcNAqah7dEoVDUqL5VnJ/L4OGj2Llz51W3lwlFIHAqGjcVQzqHcFvrwDojQQtzs/hw0hjaSwmMX3SOFuFa0s56sPjFVuzf4W9zXFV3W/Wy6iZMnMkqZrV8ke+PpJOaf3UbtabklbH+YCobD6dZm8nUJquFkROWUJy/nD9+6IBCaSLmxk8xGV9F7e6OXltOQVY6N/d/wKb6lgIFyz/9knVfrKD7Td1YvHixTSq5sxFLA4HTUSgUSNHNCPHV8MPxTMr1NUNqLQ/0b2s/YcSrs3h5iXmp8NdOf75eGMHJ/d4MeyHdetycUf1sMhmrl1U3YeJcdgnnskvw93SjQ6gvHUJ88PN0a/D1lOoMyBfySEgrIr+OZU9VJWUpypKXqebTmS1JO+uBh7eBJ6aksHfLqmo5Gok2LeZUSgUDOgQ3au8Nl+10JDrgNBxXkK+ovIKtRzPIKDK/qasn11hQu7nz9oaD/Lk9gO8+DEWvVQJngceBfTWOr6+SNYACBSF+GsIDPAj39yDM3wMPBxq36g1GMgq1pBWUc7GgnJMp2Xh51e6qq+2alMq78fLbQnG+mhbhWp56M5ngKF3d98Fdw6mkDMIDHC/46ujf+Zrshiy4PvD1UDOsexi/nc7heFqh3cpHXW4fwH1PT0ShgFvvySemUymfzWxBVkprzCUw56Nym4dCUU6FXo+6svV7fWXVTZhILywnvbCcA5gVg5+nGm+NGk83FWUF2bwz+QXi5i/Dy7855RVGynQGckp0NgFMtb0sLZWcxy/6it+++e+lCsMKNzBNwWicQXG+krY3FTPq9RSbxCF7TWa633E3K5a8d1lKwFkIG4HgqqNWKenfIYh+7YNQKRQk/rXPpvJR9Qc6OFJH687jgQWAApiCQb+HCl03Qlq2rreSdW2YMFFQpic1v4wzWcW8/958juzfx9J33+F0ZjHJuaVkF2sdjmK0LAX2bV2Dxsu7ssx4MJg2A5ZZ7yzOHQ2yUQJgP0ejU3QIHVpHXdY1OQsxIxA0Gp3C/Vi0/QuK8rIJiW7D45Pms2vjKpsH2nbKvBJzj8tPgU5APBkXFrLohVGo3Srsdml2hJq5/3V3fnbkfDP3o3L7FIO+OZCJym0sXXvDfU//YHccS47GgyNHcXrnBrIymy6KVigCQaNQvclKxoUzLHrh4RoPX82lwwF8/AdTkPMCJuMrwGTcNGO5f1wWJpO5IY6j1DaVr2pvuJLz1e6R+Ab8j7zMPhj0AL+hcnsSY8UFPLwerXX5MmbaYtqH+NK/QxCq4Q0rPtpQxNJA0CjYyywdcN8wXlm2weY4e0VTNZ4KME1Apb4LkNFrg1j//o0snxxN6rnacx1qcz1apvKOds6yYDl/17efk/jXPvRaLUrVaCp0B8jL7IObxkhozDJiB3/ES/95t97lixTdjIE3BrtEI54GzQgkSVIC44DZQD9Zlo86RSrBdUfVzFIPDw+0Wi2tw1vwcGxb9mcYKKviYqzel/Fo/A7r5/jN75F0UqIg6yXOHvHmPy+0pusdhfQfmY23f4pNG7a6XI9gTnZ6cdGqGr0fqzPnsd42S4H9OzZirtj8NkZDLADe/gd4cZEvzcP6AH0Aam0lp1Ao6NOuBZ3D/a7oXl4NGro06IbZt9MIfYUE1zqWzNKqTVYi/DXcEBXID8czSCswLx2qpjFXL5r68IvmPP3SokR+XBlE/OZA/trpz187/QkMySE3w1Tjwaea8a+667G2B9bCy8u+4+eVH3Dw183ArcBbwIDKvRkoVdOZ8dXLKBR1h1YDuKmUDOoU4tTMQWfQIEUgy/JBAEmSLus8R1oyiVZYDcfV5Fu6dKn19zfeeAO4JGPvCDX7TXA0rbTWll82KKHnfec4d3wxqWfuw2R8mtyMWwEZk3E/8Dnmkl85KJRKq2KwuB5Vbh6oNF6UlJTYbdNWFYPJn5yM+zBXae5ZuTUfpeo9OvQ8wr1PPUNpaf1JUD4aFX1aBeBlKCYzs7j+a3QQZ/yd61UEkiRtw1y0vjrTZVneeCVf6miQS1MHwziCq8vo6vLBJRmHhITQOaeUn05kUVpP/D7Atk/fJTVxOzf306At3cfxPySMhjGYC6jeAiwEfsBk3AHsAg5TUdmgpby4AG9vb+s4Vdu0FeRk8dnMxdzc722ST4ZxZI8vFbpeAChVJRgNS4CFmIx5+Dd7lNDI6HplDfP3YHDnULycUE3IHg39O9erCGRZvqdB3yAQXAbRzb0YKUXw0wlzUpE9qrvvzGt2gC9RuU3HoL8XtfszVOjuAh6o/AEowifgLK06h9As2I3JQ97FaMwGIoHZxG8OJH5zIhALrCfl9KXv9PD+k0FjfDi1fyL+LXyJHfx/9doWLHQM86VvuyCXMArWhnAfClwOb42aB7qG8ldKIXvO5tQoYmovKs/L15+2N8Vy50OjKx/QuTz0QiQr5/3M2SMa4E6gDcX53Tiy2zLS29ROBrAHc2TjOrSlF8i48ChPvTnfekR9tgWlQkHvts1r1Hh0RRrqNWgGvAD4A89IkrRKluW9TpFM8LdGoVBwU5Q/LQM9+Skhy5qrADVdjHptOUZvXwY/9Wo1418FXr7ruG1IC2IH+7Nz/UKyU5vR894XKM5XU5yvJuGPv8hK+QulqgCjIRPIBg4AZ2zkMZkuL/DIz8ONQZ1CCPGrv5SbK9BQY2EeMKfyRyBwOoHe7jxyczgHkgv443yudXZQs8tSojXrzxL488SUhTYeiJETOgBQmJto3Z+bPpfs1J8xGurvwVg1L6Iu2gR50799EBoHEpxcBRFQJHB5lEoFUnQAI6VIwvzNCTk1uyxdKv0954l+NjULqlM1XXjMtMVM/d/PNmXBFEoV/kG2lYDs1VG0YAlcKi/IoU+7IHOF52tICYCwEQiuIcyzgwhOpBex50xuDVuBBZPJ/HavPpWvK8dAuvtBm2hGpVLFbUNGkplyjsLcLPwCgwiObGXXOGhRLGe3f0GXB9+/+jfiKiAUgeCao0OoL61bePPnBR9+8vaxCRVuEd6SgpxMuzkE9oyMlv3ffjDHJpqxKC/bamsoKSmxuhqrUl2xfPrfj/n0vx/j4eHhcjEc9SEUgeCaxF2t5PY2zQlUlDHk0VG0v+tB9mxewwl5V605BPZSfy37q0cz1vv9KiVrftrHVx/M5ftNmygrK8PT05OhQ4cyb968q3bdVwuhCATXNOu+MTf+KNZWcOCOW/n3s0/SQbrD5s1elep5DI7EAVRFrVTSNdKPm6MC8HRXsfVLf5v8iWu1M5dQBILrAh+NmjtvaMHubRs4nVXCifQiwlt3qHHc5b75LbTw0XBjmC/tgn1seg3ay5+4FhGKQHBdoXFT0Tncj87hfuSX6jmdWczFgnIyCsrROeAitKBUKAjyMdc7bK7yoGOrCLvHVa0svHhxzbbv1wpCEQiuWwK83OgR04wegNFoIrtER0ahllJdBaU6c+qzrsKIu1qJu1qJh1qFl7uKYF8NIX4a3CrbqjmSJHetIxSB4G+BUqkg2FdDsO+1EenX2IiAIoFAIBSBQCAQikAgECAUgUAgQCgCgUCAUAQCgQChCAQCAUIRCAQChCIQCAQIRSAQCGh48dJFmLscFWPuevSKLMvXZvqVQPA3pqEzghJZlt+QZXkucBB4wwkyCQSCRqZBikCW5anVxnJeHyeBQNBoOKXlmSRJAcBA4GFHvlT0PmwcXF0+EDI6g0bpfVhfyzNJkvyBZcBYWZZzHflS0fuw8XB1+UDI6AwaKl+DlgaSJLUAPgAmyrJ8TpIkh2YEAoHAtWhoYZLtlWOsrGyNXgSsa6hQAoGgcWloy7ObnSWIQCBoOkRAkUAgEIpAIBAIRSAQCBCKQCAQIBSBQCBAKAKBQIBQBAKBAKEIBAIBQhEIBAKEIhAIBAhFIBAIEIpAIBAgFIFAIEAoAoFAgFAEAoEAoQgEAgFCEQgEAoQiEAgECEUgEAhoeMuzl4EuwCngdmCeLMvxzhBMIBA0Hg2dEWiAF2VZng98BsxqsEQCgaDRaWgV4/lVPrYFjjdMHIFA0BQ0uOWZJEmhwBSgOzDMkS+dMWPGZQkpEAiuLgqTyeSUgSRJ6ofZRnCrUwYUCASNRkNbnk2s8vEc0Lph4ggEgqagoS3PWkqS9C6QDXQDnm64SAKBoLFx2tJAIBBcu4iAIoFAIBSBQCBouI2gwUiSNACz2zETMMmyPLPafg9gIXARuAGzZ+KUC8k3GQgF0oFbMLtVTzSWfI7IWOW4fwBfAr6yLBc3ooiO3EcF8GLlxxggQJblsS4kXyvM/4d/AjcBq2RZ3tiI8oUCc4Busiz3sLNfCbwNFAPRwCeyLO91dPwmnRFIkuQFLAdelWX5TaCrJEn9qx32CpAky/JcYBHwiYvJ5wO8JsvyO8A6YEFjyXcZMiJJUkfgxsaUrcp3OyLjE0C+LMtLZFl+DfiPi8k3Cdgty/I84B3g3caSr5LewAZAUcv+RwE/WZbnAJOBLyRJUjk6eFMvDW4DLsiyrK38/DswpNoxQ4B4AFmWjwDdJEnycxX5ZFmeJsuyxeKqxKyRG5N6Zaz8R58E2J0pNAKO/J3/AQRKkvSSJEmWN5sryZcBBFX+HgTsbyTZAJBl+RugqI5Dqj4nuUA50MnR8ZtaEQRje3GFldsu95irhcPfLUmSOzAGmNoIclXFERnfAmbLsqxrNKlscUTGaMxvtCWY81Z+uJw3WgNxRL73gJ6SJL0HTAc+bSTZHKVBz0lTK4JMwLfKZ7/KbZd7zNXCoe+uVAIfAm/IsnymkWSzUKeMkiRFAc2ARyVJiqvc/JokSVLjiejQfSwE9gFU2oD8gKhGkc4x+T4DPq5ctjwErJYkKbBxxHOIBj0nTa0I4oFoSZI0lZ9vBzZLkhRYZfq/GfPUDUmSugB/ybJc6CrySZLkCXwEvCfL8n5Jkh5uJNkcklGW5WRZlp+UZXle5fqWSlllV5GxctsOKiNTK7epMBtgXUW+KCCt8vc8wEjT29i8JUmyLFeqPieBgAdwzNGxmjygSJKku4FHgCxAL8vyTEmS5gO5sizPq3zQFmL+I7QF3m5kr0F98q0HOgOplad427PqNqWMlccEAc8Csyt/PpJl+aKryChJkj8wH7gAtAHWybK8xYXk643ZcH0AaAXsl2V5eSPKdxcwGhiEefb5LjAW6CLL8nOVXoO5QCnQEvi/y/EaNLkiEAgETU9TLw0EAoELIBSBQCAQikAgEAhFIBAIEIpAIBAgFIFAIEAoAoFAAPw/rSwgRXJs16UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 288x216 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "with torch.no_grad():\n",
    "    # Initialize plot\n",
    "    f, ax = plt.subplots(1, 1, figsize=(4, 3))\n",
    "\n",
    "    # Plot training data as black stars\n",
    "    ax.plot(train_x.numpy(), train_y.numpy(), 'k*')\n",
    "    # Plot predictive means as blue line\n",
    "    ax.plot(test_x.numpy(), mean.numpy(), 'b')\n",
    "    # Shade between the lower and upper confidence bounds\n",
    "    ax.fill_between(test_x.numpy(), lower.numpy(), upper.numpy(), alpha=0.5)\n",
    "    ax.set_ylim([-3, 3])\n",
    "    ax.legend(['Observed Data', 'Mean', 'Confidence'])"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}