{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# QPyTorch Regression Tutorial\n", "\n", "## Introduction\n", "\n", "In this notebook, we demonstrate many of the design features of QPyTorch using the simplest example, training an RBF kernel q-exponential process on a simple function. We'll be modeling the function\n", "\n", "$$\n", "\\begin{align}\n", "y &= \\sin(2\\pi x) + \\epsilon \\\\\n", " \\epsilon &\\sim \\mathcal{Q}(0, 0.04) \n", "\\end{align}\n", "$$\n", "\n", "with 100 training examples, and testing on 51 test examples.\n", "\n", "**Note:** this notebook is not necessarily intended to teach the mathematical background of Gaussian/q-exponential processes, but rather how to train a simple one and make predictions in QPyTorch. For a mathematical treatment, Chapter 2 of Gaussian Processes for Machine Learning provides a very thorough introduction to GP regression (this entire text is highly recommended): http://www.gaussianprocess.org/gpml/chapters/RW2.pdf" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import math\n", "import torch\n", "import qpytorch\n", "from matplotlib import pyplot as plt\n", "\n", "%matplotlib inline\n", "%load_ext autoreload\n", "%autoreload 2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Set up training data\n", "\n", "In the next cell, we set up the training data for this example. We'll be using 100 regularly spaced points on [0,1] which we evaluate the function on and add Gaussian noise to get the training labels." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# Training data is 100 points in [0,1] inclusive regularly spaced\n", "train_x = torch.linspace(0, 1, 100)\n", "# True function is sin(2*pi*x) with Gaussian noise\n", "train_y = torch.sin(train_x * (2 * math.pi)) + torch.randn(train_x.size()) * math.sqrt(0.04)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setting up the model\n", "\n", "The next cell demonstrates the most critical features of a user-defined q-exponential process model in QPyTorch. Building a QEP model in QPyTorch is different in a number of ways.\n", "\n", "First in contrast to many existing QEP packages, we do not provide full QEP models for the user. Rather, we provide *the tools necessary to quickly construct one*. This is because we believe, analogous to building a neural network in standard PyTorch, it is important to have the flexibility to include whatever components are necessary. As can be seen in more complicated examples, this allows the user great flexibility in designing custom models.\n", "\n", "For most QEP regression models, you will need to construct the following QPyTorch objects:\n", "\n", "1. A **QEP Model** (`qpytorch.models.ExactQEP`) - This handles most of the inference.\n", "1. A **Likelihood** (`qpytorch.likelihoods.QExponentialLikelihood`) - This is the most common likelihood used for QEP regression.\n", "1. A **Mean** - This defines the prior mean of the QEP.(If you don't know which mean to use, a `qpytorch.means.ConstantMean()` is a good place to start.)\n", "1. A **Kernel** - This defines the prior covariance of the QEP.(If you don't know which kernel to use, a `qpytorch.kernels.ScaleKernel(qpytorch.kernels.RBFKernel())` is a good place to start).\n", "1. A **MultivariateQExponential** Distribution (`qpytorch.distributions.MultivariateQExponential`) - This is the object used to represent multivariate q-exponential distributions.\n", " \n", " \n", "### The QEP Model\n", " \n", "The components of a user built (Exact, i.e. non-variational) QEP model in QPyTorch are, broadly speaking:\n", "\n", "1. An `__init__` method that takes the training data and a likelihood, and constructs whatever objects are necessary for the model's `forward` method. This will most commonly include things like a mean module and a kernel module.\n", "\n", "2. A `forward` method that takes in some $n \\times d$ data `x` and returns a `MultivariateQExponential` with the *prior* mean and covariance evaluated at `x`. In other words, we return the vector $\\mu(x)$ and the $n \\times n$ matrix $K_{xx}$ representing the prior mean and covariance matrix of the QEP. \n", "\n", "This specification leaves a large amount of flexibility when defining a model. For example, to compose two kernels via addition, you can either add the kernel modules directly:\n", "\n", "```python\n", "self.covar_module = ScaleKernel(RBFKernel() + LinearKernel())\n", "```\n", "\n", "Or you can add the outputs of the kernel in the forward method:\n", "\n", "```python\n", "covar_x = self.rbf_kernel_module(x) + self.white_noise_module(x)\n", "```\n", "\n", "\n", "### The likelihood\n", "\n", "The simplest likelihood for regression is the `qpytorch.likelihoods.QExponentialLikelihood`. This assumes a homoskedastic noise model (i.e. all inputs have the same observational noise).\n", "\n", "There are other options for exact QEP regression, such as the [FixedNoiseQExponentialLikelihood](https://qepytorch.readthedocs.io/en/stable/likelihoods.html#fixednoiseqexponentiallikelihood), which assigns a different observed noise value to different training inputs." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "# We will use the simplest form of QEP model, exact inference\n", "POWER = 1.0\n", "class ExactQEPModel(qpytorch.models.ExactQEP):\n", " def __init__(self, train_x, train_y, likelihood):\n", " super(ExactQEPModel, self).__init__(train_x, train_y, likelihood)\n", " self.power = torch.tensor(POWER)\n", " self.mean_module = qpytorch.means.ConstantMean()\n", " self.covar_module = qpytorch.kernels.ScaleKernel(qpytorch.kernels.RBFKernel())\n", " \n", " def forward(self, x):\n", " mean_x = self.mean_module(x)\n", " covar_x = self.covar_module(x)\n", " return qpytorch.distributions.MultivariateQExponential(mean_x, covar_x, power=self.power)\n", "\n", "# initialize likelihood and model\n", "likelihood = qpytorch.likelihoods.QExponentialLikelihood(power=torch.tensor(POWER))\n", "model = ExactQEPModel(train_x, train_y, likelihood)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Model modes\n", "\n", "Like most PyTorch modules, the `ExactQEP` has a `.train()` and `.eval()` mode.\n", "- `.train()` mode is for optimizing model hyperameters.\n", "- `.eval()` mode is for computing predictions through the model posterior." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Training the model\n", "\n", "In the next cell, we handle using Type-II MLE to train the hyperparameters of the q-exponential process.\n", "\n", "The most obvious difference here compared to many other QEP implementations is that, as in standard PyTorch, the core training loop is written by the user. In QPyTorch, we make use of the standard PyTorch optimizers as from `torch.optim`, and all trainable parameters of the model should be of type `torch.nn.Parameter`. Because QEP models directly extend `torch.nn.Module`, calls to methods like `model.parameters()` or `model.named_parameters()` function as you might expect coming from PyTorch.\n", "\n", "In most cases, the boilerplate code below will work well. It has the same basic components as the standard PyTorch training loop:\n", "\n", "1. Zero all parameter gradients\n", "2. Call the model and compute the loss\n", "3. Call backward on the loss to fill in gradients\n", "4. Take a step on the optimizer\n", "\n", "However, defining custom training loops allows for greater flexibility. For example, it is easy to save the parameters at each step of training, or use different learning rates for different parameters (which may be useful in deep kernel learning for example)." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Iter 1/80 - Loss: 1.678 lengthscale: 0.693 noise: 0.693\n", "Iter 2/80 - Loss: 1.648 lengthscale: 0.644 noise: 0.644\n", "Iter 3/80 - Loss: 1.614 lengthscale: 0.598 noise: 0.598\n", "Iter 4/80 - Loss: 1.573 lengthscale: 0.555 noise: 0.554\n", "Iter 5/80 - Loss: 1.524 lengthscale: 0.514 noise: 0.513\n", "Iter 6/80 - Loss: 1.465 lengthscale: 0.476 noise: 0.474\n", "Iter 7/80 - Loss: 1.396 lengthscale: 0.440 noise: 0.437\n", "Iter 8/80 - Loss: 1.321 lengthscale: 0.406 noise: 0.402\n", "Iter 9/80 - Loss: 1.246 lengthscale: 0.373 noise: 0.369\n", "Iter 10/80 - Loss: 1.178 lengthscale: 0.343 noise: 0.338\n", "Iter 11/80 - Loss: 1.122 lengthscale: 0.315 noise: 0.310\n", "Iter 12/80 - Loss: 1.079 lengthscale: 0.289 noise: 0.284\n", "Iter 13/80 - Loss: 1.045 lengthscale: 0.266 noise: 0.259\n", "Iter 14/80 - Loss: 1.017 lengthscale: 0.247 noise: 0.237\n", "Iter 15/80 - Loss: 0.991 lengthscale: 0.230 noise: 0.216\n", "Iter 16/80 - Loss: 0.969 lengthscale: 0.215 noise: 0.197\n", "Iter 17/80 - Loss: 0.949 lengthscale: 0.203 noise: 0.180\n", "Iter 18/80 - Loss: 0.930 lengthscale: 0.192 noise: 0.164\n", "Iter 19/80 - Loss: 0.913 lengthscale: 0.183 noise: 0.150\n", "Iter 20/80 - Loss: 0.895 lengthscale: 0.176 noise: 0.137\n", "Iter 21/80 - Loss: 0.879 lengthscale: 0.171 noise: 0.125\n", "Iter 22/80 - Loss: 0.862 lengthscale: 0.166 noise: 0.114\n", "Iter 23/80 - Loss: 0.845 lengthscale: 0.163 noise: 0.104\n", "Iter 24/80 - Loss: 0.827 lengthscale: 0.161 noise: 0.095\n", "Iter 25/80 - Loss: 0.810 lengthscale: 0.159 noise: 0.087\n", "Iter 26/80 - Loss: 0.792 lengthscale: 0.159 noise: 0.079\n", "Iter 27/80 - Loss: 0.773 lengthscale: 0.159 noise: 0.072\n", "Iter 28/80 - Loss: 0.754 lengthscale: 0.160 noise: 0.066\n", "Iter 29/80 - Loss: 0.735 lengthscale: 0.162 noise: 0.060\n", "Iter 30/80 - Loss: 0.715 lengthscale: 0.165 noise: 0.055\n", "Iter 31/80 - Loss: 0.694 lengthscale: 0.168 noise: 0.050\n", "Iter 32/80 - Loss: 0.674 lengthscale: 0.172 noise: 0.046\n", "Iter 33/80 - Loss: 0.653 lengthscale: 0.177 noise: 0.042\n", "Iter 34/80 - Loss: 0.631 lengthscale: 0.182 noise: 0.038\n", "Iter 35/80 - Loss: 0.610 lengthscale: 0.188 noise: 0.035\n", "Iter 36/80 - Loss: 0.588 lengthscale: 0.194 noise: 0.032\n", "Iter 37/80 - Loss: 0.566 lengthscale: 0.201 noise: 0.029\n", "Iter 38/80 - Loss: 0.544 lengthscale: 0.209 noise: 0.027\n", "Iter 39/80 - Loss: 0.522 lengthscale: 0.217 noise: 0.024\n", "Iter 40/80 - Loss: 0.500 lengthscale: 0.226 noise: 0.022\n", "Iter 41/80 - Loss: 0.479 lengthscale: 0.236 noise: 0.020\n", "Iter 42/80 - Loss: 0.457 lengthscale: 0.245 noise: 0.019\n", "Iter 43/80 - Loss: 0.436 lengthscale: 0.256 noise: 0.017\n", "Iter 44/80 - Loss: 0.414 lengthscale: 0.267 noise: 0.016\n", "Iter 45/80 - Loss: 0.393 lengthscale: 0.278 noise: 0.014\n", "Iter 46/80 - Loss: 0.373 lengthscale: 0.289 noise: 0.013\n", "Iter 47/80 - Loss: 0.352 lengthscale: 0.301 noise: 0.012\n", "Iter 48/80 - Loss: 0.332 lengthscale: 0.313 noise: 0.011\n", "Iter 49/80 - Loss: 0.312 lengthscale: 0.326 noise: 0.010\n", "Iter 50/80 - Loss: 0.293 lengthscale: 0.338 noise: 0.009\n", "Iter 51/80 - Loss: 0.273 lengthscale: 0.351 noise: 0.008\n", "Iter 52/80 - Loss: 0.254 lengthscale: 0.364 noise: 0.008\n", "Iter 53/80 - Loss: 0.236 lengthscale: 0.377 noise: 0.007\n", "Iter 54/80 - Loss: 0.217 lengthscale: 0.390 noise: 0.006\n", "Iter 55/80 - Loss: 0.200 lengthscale: 0.404 noise: 0.006\n", "Iter 56/80 - Loss: 0.183 lengthscale: 0.416 noise: 0.005\n", "Iter 57/80 - Loss: 0.167 lengthscale: 0.429 noise: 0.005\n", "Iter 58/80 - Loss: 0.151 lengthscale: 0.440 noise: 0.004\n", "Iter 59/80 - Loss: 0.136 lengthscale: 0.451 noise: 0.004\n", "Iter 60/80 - Loss: 0.121 lengthscale: 0.461 noise: 0.004\n", "Iter 61/80 - Loss: 0.107 lengthscale: 0.469 noise: 0.003\n", "Iter 62/80 - Loss: 0.093 lengthscale: 0.476 noise: 0.003\n", "Iter 63/80 - Loss: 0.079 lengthscale: 0.482 noise: 0.003\n", "Iter 64/80 - Loss: 0.066 lengthscale: 0.486 noise: 0.003\n", "Iter 65/80 - Loss: 0.053 lengthscale: 0.488 noise: 0.002\n", "Iter 66/80 - Loss: 0.040 lengthscale: 0.490 noise: 0.002\n", "Iter 67/80 - Loss: 0.028 lengthscale: 0.490 noise: 0.002\n", "Iter 68/80 - Loss: 0.017 lengthscale: 0.489 noise: 0.002\n", "Iter 69/80 - Loss: 0.005 lengthscale: 0.488 noise: 0.002\n", "Iter 70/80 - Loss: -0.005 lengthscale: 0.487 noise: 0.002\n", "Iter 71/80 - Loss: -0.015 lengthscale: 0.485 noise: 0.002\n", "Iter 72/80 - Loss: -0.025 lengthscale: 0.483 noise: 0.001\n", "Iter 73/80 - Loss: -0.033 lengthscale: 0.481 noise: 0.001\n", "Iter 74/80 - Loss: -0.042 lengthscale: 0.479 noise: 0.001\n", "Iter 75/80 - Loss: -0.049 lengthscale: 0.478 noise: 0.001\n", "Iter 76/80 - Loss: -0.057 lengthscale: 0.477 noise: 0.001\n", "Iter 77/80 - Loss: -0.063 lengthscale: 0.477 noise: 0.001\n", "Iter 78/80 - Loss: -0.070 lengthscale: 0.477 noise: 0.001\n", "Iter 79/80 - Loss: -0.076 lengthscale: 0.477 noise: 0.001\n", "Iter 80/80 - Loss: -0.081 lengthscale: 0.478 noise: 0.001\n" ] } ], "source": [ "# this is for running the notebook in our testing framework\n", "import os\n", "smoke_test = ('CI' in os.environ)\n", "training_iter = 2 if smoke_test else 80\n", "\n", "\n", "# Find optimal model hyperparameters\n", "model.train()\n", "likelihood.train()\n", "\n", "# Use the adam optimizer\n", "optimizer = torch.optim.Adam(model.parameters(), lr=0.1) # Includes GaussianLikelihood parameters\n", "\n", "# \"Loss\" for QEPs - the marginal log likelihood\n", "mll = qpytorch.mlls.ExactMarginalLogLikelihood(likelihood, model)\n", "\n", "for i in range(training_iter):\n", " # Zero gradients from previous iteration\n", " optimizer.zero_grad()\n", " # Output from model\n", " output = model(train_x)\n", " # Calc loss and backprop gradients\n", " loss = -mll(output, train_y)\n", " loss.backward()\n", " print('Iter %d/%d - Loss: %.3f lengthscale: %.3f noise: %.3f' % (\n", " i + 1, training_iter, loss.item(),\n", " model.covar_module.base_kernel.lengthscale.item(),\n", " model.likelihood.noise.item()\n", " ))\n", " optimizer.step()" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Make predictions with the model\n", "\n", "In the next cell, we make predictions with the model. To do this, we simply put the model and likelihood in eval mode, and call both modules on the test data.\n", "\n", "Just as a user defined QEP model returns a `MultivariateQExponential` containing the prior mean and covariance from forward, a trained QEP model in eval mode returns a `MultivariateQExponential` containing the posterior mean and covariance.\n", "\n", "If we denote a test point (`test_x`) as `x*` with the true output being `y*`, then `model(test_x)` returns the model posterior distribution `p(f* | x*, X, y)`, for training data `X, y`. This posterior is the distribution over the function we are trying to model, and thus quantifies our model uncertainty.\n", "\n", "In contrast, `likelihood(model(test_x))` gives us the posterior predictive distribution `p(y* | x*, X, y)` which is the probability distribution over the predicted output value. Recall in our problem setup \n", "\n", "$$\n", "\\begin{align}\n", "y &= \\sin(2\\pi x) + \\epsilon\n", "\\end{align}\n", "$$\n", "\n", "where 𝜖 is the likelihood noise for each observation. By including the _likelihood noise_ which is the noise in your observation (e.g. due to noisy sensor), the prediction is over the observed value of the test point.\n", "\n", "Thus, getting the predictive mean and variance, and then sampling functions from the QEP at the given test points could be accomplished with calls like:\n", "\n", "```python\n", "f_preds = model(test_x)\n", "y_preds = likelihood(model(test_x))\n", "\n", "f_mean = f_preds.mean\n", "f_var = f_preds.variance\n", "f_covar = f_preds.covariance_matrix\n", "f_samples = f_preds.sample(sample_shape=torch.Size(1000,))\n", "```\n", "\n", "The `qpytorch.settings.fast_pred_var` context is not needed, but here we are giving a preview of using one of our cool features, getting faster predictive distributions using [LOVE](https://arxiv.org/abs/1803.06058)." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "# Get into evaluation (predictive posterior) mode\n", "model.eval()\n", "likelihood.eval()\n", "\n", "# Test points are regularly spaced along [0,1]\n", "# Make predictions by feeding model through likelihood\n", "with torch.no_grad(), qpytorch.settings.fast_pred_var():\n", " test_x = torch.linspace(0, 1, 51)\n", " observed_pred = likelihood(model(test_x))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Plot the model fit\n", "\n", "In the next cell, we plot the mean and confidence region of the Gaussian process model. The `confidence_region` method is a helper method that returns 2 standard deviations above and below the mean." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWgAAAEYCAYAAABxx2wUAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAWI5JREFUeJzt3Xd8k9X+wPHPk7RJ996lg72HFEGGgqICFxEciIoIiuhVUAEH+vMqOLigeN24cKAMURAEAUFECgKyN5RRoLR07z2T8/sjNDZt2ialKaGc9+uVV0medZ7QfnNy1lcRQggkSZIku6O60gWQJEmSzJMBWpIkyU7JAC1JkmSnZICWJEmyUzJAS5Ik2SkZoCVJkuyUDNCSJEl2SgZoSZIkOyUDtCRJkp2SAVqSJMlO2TRAf/bZZ3Tr1g0PDw88PDzo27cvv/32my0vKUmS1GwotlyL49dff0WtVtO2bVuEEHz33XfMmzePgwcP0rlzZ1tdVpIkqVmwaYA2x8fHh3nz5jFx4sSmvKwkSdJVx6GpLqTT6Vi+fDmFhYX07dvX7D6lpaWUlpYan+v1erKysvD19UVRlKYqqiRJks0IIcjPzyckJASVqp5WZmFjR44cEa6urkKtVgtPT0+xbt26WvedOXOmAORDPuRDPpr9IyEhod74afMmjrKyMuLj48nNzWXFihV89dVXbN26lU6dOtXYt3oNOjc3l/DwcBISEvDw8LBlMSVJkppEXl4eYWFh5OTk4OnpWee+Td4Gfeutt9K6dWu++OKLevfNy8vD09OT3NxcGaAlSWoWrIlrTT4OWq/Xm9SSJUmSJPNs2kn48ssvM2zYMMLDw8nPz2fp0qVER0ezceNGW15WkiSpWbBpgE5LS+Phhx8mOTkZT09PunXrxsaNG7nttttseVlJkqRmwaYB+uuvv7bl6SWpwfR6PWVlZVe6GFIz5OjoiFqtbpRzNdk4aEmyF2VlZZw/fx69Xn+liyI1U15eXgQFBV32/A0ZoKVrihCC5ORk1Go1YWFh9U8UkCQrCCEoKioiLS0NgODg4Ms6nwzQ0jWloqKCoqIiQkJCcHFxudLFkZohZ2dnwNAHFxAQcFnNHbL6IF1TdDodABqN5gqXRGrOKj/8y8vLL+s8MkBL1yS5totkS431+yUDtCRJkp2SAVqSmpnIyEg++OCDK12MRtPc7scaMkBL0lUiISGBRx99lJCQEDQaDRERETz77LNkZmZe6aJdUbNmzUJRFBRFwcHBAT8/P2666SY++OADq5eViI6ORlEUcnJybFNYK8kALUkNtG/fPm655Rb27dtn82udO3eOXr16cebMGX744QdiY2P5/PPP2bx5M3379iUrK8vmZaiNTqe74mPKO3fuTHJyMvHx8WzZsoXRo0czZ84c+vXrR35+/hUt2+WQAVqSGuj7779ny5YtLFq0yObXmjx5MhqNht9//52BAwcSHh7OsGHD+OOPP0hMTOSVV14x2T8/P58HHngAV1dXQkNDmT9/vnGbEIJZs2YRHh6OVqslJCSEZ555xri9tLSU559/ntDQUFxdXenTpw/R0dHG7QsXLsTLy4s1a9bQqVMntFotX331FU5OTjVqns8++yy33HKL8fn27du58cYbcXZ2JiwsjGeeeYbCwkLj9rS0NEaMGIGzszMtW7ZkyZIlFr0/Dg4OBAUFERISQteuXXn66afZunUrx44d4+233zbut2jRInr16oW7uztBQUE8+OCDxjHLcXFx3HzzzQB4e3ujKAoTJkwAYMOGDQwYMAAvLy98fX254447OHv2rEVluyyNsiq/jeTm5gpA5ObmXumiSM1EcXGxOHHihCguLm7Q8XFxcWLfvn1i//79IiAgQAAiICBA7N+/X+zbt0/ExcU1comFyMzMFIqiiP/+979mt0+aNEl4e3sLvV4vhBAiIiJCuLu7izlz5ohTp06Jjz76SKjVavH7778LIYRYvny58PDwEOvXrxcXLlwQu3fvFl9++aXxfI899pjo16+f2LZtm4iNjRXz5s0TWq1WnD59WgghxLfffiscHR1Fv379xI4dO8TJkydFQUGBCAwMFF999ZXxPBUVFSavxcbGCldXV/H++++L06dPix07dojrrrtOTJgwwXjMsGHDRPfu3cXff/8t9u3bJ/r16yecnZ3F+++/X+v7M3PmTNG9e3ez20aOHCk6duxofP7111+L9evXi7Nnz4q///5b9O3bVwwbNsxY3p9//lkA4tSpUyI5OVnk5OQIIYRYsWKF+Pnnn8WZM2fEwYMHxYgRI0TXrl2FTqcze926fs+siWsyQEvXlMsN0FTJiKEoisnPykdj27VrlwDEqlWrzG5/7733BCBSU1OFEIYAPXToUJN9xowZYwxE//vf/0S7du1EWVlZjXNduHBBqNVqkZiYaPL64MGDxcsvvyyEMARoQBw6dMhkn2effVbccsstxucbN24UWq1WZGdnCyGEmDhxonj88cdNjvnrr7+ESqUSxcXF4tSpUwIQe/bsMW6PiYkRQIMD9IwZM4Szs3Otx+7du1cAIj8/XwghxJYtWwRgLHNt0tPTBSCOHj1qdntjBWjZxCFJVli8eDEODoYJuOJSrovKnw4ODixevNhm1xZW5Naonvezb9++xMTEADB69GiKi4tp1aoVkyZNYtWqVVRUVABw9OhRdDod7dq1w83NzfjYunWryVd6jUZDt27dTK4xduxYoqOjSUpKAmDJkiUMHz4cLy8vAA4fPszChQtNzjtkyBD0ej3nz58nJiYGBwcHoqKijOfs0KGD8fiGEEKYjEnev38/I0aMIDw8HHd3dwYOHAhAfHx8nec5c+YMDzzwAK1atcLDw4PIyEiLjrtccqq3JFlh7NixdOzY0SSIVNq9ezc9e/Zs9Gu2adMGRVGIiYnhrrvuqrE9JiYGb29v/P39LTpfWFgYp06d4o8//mDTpk089dRTzJs3j61bt1JQUIBarWb//v01pii7ubkZ/+3s7FxjMsb1119P69atWbZsGU8++SSrVq1i4cKFxu0FBQU88cQTJu3dlcLDwzl9+rRF5bdGTEwMLVu2BKCwsJAhQ4YwZMgQlixZgr+/P/Hx8QwZMqTelQ1HjBhBREQECxYsICQkBL1eT5cuXWy+IqIM0JLUQCqVCr1eb/xpK76+vtx22218+umnTJs2zbjWA0BKSgpLlizh4YcfNgmYu3btMjnHrl276Nixo/G5s7MzI0aMYMSIEUyePJkOHTpw9OhRrrvuOnQ6HWlpadx4441Wl3Xs2LEsWbKEFi1aoFKpGD58uHFbz549OXHiBG3atDF7bIcOHaioqGD//v1cf/31AJw6darBQ95OnjzJhg0bePnll43PMzMzmTt3LmFhYQA1RuBULgFQuSQAQGZmJqdOnWLBggXG92T79u0NKpO1ZBOHJFkpICCAoKAgoqKi+Pzzz4mKiiIoKIiAgACbXfOTTz6htLSUIUOGsG3bNhISEtiwYQO33XYboaGhzJ4922T/HTt28M4773D69Gnmz5/P8uXLefbZZwHDKIyvv/6aY8eOce7cORYvXoyzszMRERG0a9eOsWPH8vDDD7Ny5UrOnz/Pnj17mDNnDuvWrau3nGPHjuXAgQPMnj2be++9F61Wa9w2Y8YMdu7cyZQpUzh06BBnzpxh9erVTJkyBYD27dszdOhQnnjiCXbv3s3+/ft57LHHTD6QalNRUUFKSgpJSUkcPXqUjz/+mIEDB9KjRw9eeOEFwFBL12g0fPzxx5w7d441a9bw5ptvmpwnIiICRVFYu3Yt6enpFBQU4O3tja+vL19++SWxsbH8+eefTJ8+vd4yNYp6W6mvINlJKDW2y+0krFRSUmIcNaHX60VJSUljFK9OcXFxYvz48SIwMFA4OjqKsLAw8fTTT4uMjAyT/SIiIsTrr78uRo8eLVxcXERQUJD48MMPjdtXrVol+vTpIzw8PISrq6u44YYbxB9//GHcXlZWJl577TURGRkpHB0dRXBwsLjrrrvEkSNHhBCGTkJPT89ay9m7d28BiD///LPGtj179ojbbrtNuLm5CVdXV9GtWzcxe/Zs4/bk5GQxfPhwodVqRXh4uPj+++9FREREvZ2EXOqgVavVwsfHRwwYMEC8//77Nf5fli5dKiIjI4VWqxV9+/YVa9asEYA4ePCgcZ833nhDBAUFCUVRxPjx44UQQmzatEl07NhRaLVa0a1bNxEdHV1nx21jdRI2eVZva8is3lJjKykp4fz587Rs2RInJ6crXRypmarr98yus3pLkiRJlpEBWpIkyU7JAC1JkmSnZICWJEmyUzJAS5Ik2SkZoCVJkuyUDNCSJEl2SgZoSZIkOyUDtCRJkp2SAVqSJMlO2TRAz5kzh+uvvx53d3cCAgIYNWoUp06dsuUlJalZmjBhAoqi8O9//7vGtsmTJ5ukZ5KaD5sG6K1btzJ58mR27drFpk2bKC8v5/bbbzfJQSZJkmXCwsJYtmwZxcXFxtdKSkpYunQp4eHhV7Bkkq3YNEBv2LCBCRMm0LlzZ7p3787ChQuJj49n//79trysJDVLPXv2JCwsjJUrVxpfW7lyJeHh4Vx33XXG1/R6PXPmzKFly5Y4OzvTvXt3VqxYYdyu0+mYOHGicXv79u358MMPTa41YcIERo0axbvvvktwcDC+vr5MnjyZ8vJy29+oZNSkC/bn5uYC4OPjY3Z7aWkppaWlxud5eXlNUi7p2iUEFBVdmWu7uEC1pCT1evTRR/n2228ZO3YsAN988w2PPPKISdbtOXPmsHjxYj7//HPatm3Ltm3beOihh/D392fgwIHo9XpatGjB8uXL8fX1ZefOnTz++OMEBwdz3333Gc+zZcsWgoOD2bJlC7GxsYwZM4YePXowadKkxrh9yRL1LkjaSHQ6nRg+fLjo379/rftUXde16kOuBy01lurr9BYUCGEI003/KCiwvNzjx48XI0eOFGlpaUKr1Yq4uDgRFxcnnJycRHp6uhg5cqQYP368KCkpES4uLmLnzp0mx0+cOFE88MADtZ5/8uTJ4p577jG5XkREhKioqDC+Nnr0aDFmzBjLC30Na6z1oJusBj158mSOHTtWZ6qYl19+2SRTQV5enjE1jSRJ4O/vz/Dhw1m4cCFCCIYPH46fn59xe2xsLEVFRdx2220mx5WVlZk0g8yfP59vvvmG+Ph4iouLKSsro0ePHibHdO7c2SQvYXBwMEePHrXNjUlmNUmAnjJlCmvXrmXbtm20aNGi1v20Wq1JihxJsjUXFygouHLXbohHH33UmCZq/vz5JtsKLt3MunXrCA0NNdlW+be1bNkynn/+ef73v//Rt29f3N3dmTdvHrt37zbZ39HR0eS5oig2zb0o1WTTAC2E4Omnn2bVqlVER0cbs+tKkr1QFHB1vdKlsM7QoUMpKytDURSGDBlisq1Tp05otVri4+MZOHCg2eN37NhBv379eOqpp4yvnT171qZllhrGpgF68uTJLF26lNWrV+Pu7k5KSgoAnp6eFiWClCSpJrVaTUxMjPHfVbm7u/P8888zbdo09Ho9AwYMIDc3lx07duDh4cH48eNp27Yt33//PRs3bqRly5YsWrSIvXv3ygqUHbJpgP7ss88AGDRokMnr3377rRxUL0mXoa5cdm+++Sb+/v7MmTOHc+fO4eXlRc+ePfm///s/AJ544gkOHjzImDFjUBSFBx54gKeeeorffvutqYovWUgmjZWuKTJprNQUZNJYSZKkZk4GaEmSJDslA7QkSZKdkgFakiTJTskALUmSZKdkgJYkSbJTMkBLkiTZKRmgJUmS7JQM0JIkSXZKBmhJkkwIIXj88cfx8fFBURQOHTrEoEGDmDp1ap3HRUZG8sEHHzRJGa8VTZpRRZLs1fubTjfp9abd1q5Bx6WkpDB79mzWrVtHYmIiAQEB9OjRg6lTpzJ48OBGKduGDRtYuHAh0dHRtGrVCj8/P1auXFlj+VHJ9mSAlqSrRFxcHP3798fLy4t58+bRtWtXysvL2bhxI5MnT+bkyZONcp2zZ88SHBxMv379jK/VlqZOsi3ZxCFJV4mnnnoKRVHYs2cP99xzD+3ataNz585Mnz6dXbt2ARAfH8/IkSNxc3PDw8OD++67j9TUVOM5Zs2aRY8ePVi0aBGRkZF4enpy//33k5+fDxiSxT799NPEx8ejKAqRkZEANZo40tLSGDFiBM7OzrRs2ZIlS5bUKG9OTg6PPfYY/v7+eHh4cMstt3D48GGLywKGBLjvvPMObdq0QavVEh4ezuzZs43bExISuO+++/Dy8sLHx4eRI0cSFxfXGG+3XZABWpKuAllZWWzYsIHJkyfjaibDgJeXF3q9npEjR5KVlcXWrVvZtGkT586dY8yYMSb7nj17ll9++YW1a9eydu1atm7dyty5cwH48MMPeeONN2jRogXJycns3bvXbHkmTJhAQkICW7ZsYcWKFXz66aekpaWZ7DN69GjS0tL47bff2L9/Pz179mTw4MFkZWVZVBYwpMGbO3cur776KidOnGDp0qUEBgYCUF5ezpAhQ3B3d+evv/5ix44duLm5GRMaNAeyiUOSrgKxsbEIIejQoUOt+2zevJmjR49y/vx5Yy7P77//ns6dO7N3716uv/56wFArXbhwIe7u7gCMGzeOzZs3M3v2bDw9PXF3d0etVhMUFGT2OqdPn+a3335jz549xnN+/fXXdOzY0bjP9u3b2bNnD2lpacZUW++++y6//PILK1as4PHHH6+3LPn5+Xz44Yd88sknjB8/HoDWrVszYMAAAH788Uf0ej1fffUVyqX06N9++y1eXl5ER0dz++23N+Cdti8yQEvSVcCSZdtjYmIICwszSbTcqVMnvLy8iImJMQbTyMhIY0AEQzLY6rXf+q7j4OBAVFSU8bUOHTrg5eVlfH748GEKCgrw9fU1Oba4uNgkvVZdZYmJiaG0tLTWzs/Dhw8TGxtrcjwY1mJuLim8ZIBuRkordGQVllFYqqO4TEdRWQVFZTqKynRU6PWoFOXSw5AAVK1ScNWq8XByNDycHXB3ckStUq70rUjVtG3bFkVRGqUjsCmSwRYUFBAcHEx0dHSNbVUDeV1lqS8tXkFBAVFRUWbbv/39/a0vtB2SAfoqVVKuIyW3hLT8UtLzS0nLLyG3uJzLzY+jKODh5EiwpxNBnk4Eezrj766VQfsK8/HxYciQIcyfP59nnnmmRjt0Tk4OHTt2JCEhgYSEBGMt+sSJE+Tk5NCpU6dGK0uHDh2oqKhg//79xlr5qVOnyMnJMe7Ts2dPUlJScHBwMHY0Wqtt27Y4OzuzefNmHnvssRrbe/bsyY8//khAQECzzbgkA/RVJKOglHPphZzPKCA5t6TeYJxw+ii/LpjHiEkvENauq0XXEAJyi8vJLS7nZIqhN91RrRDg7kSErwttAtzwddNe7q1IDTB//nz69+9P7969eeONN+jWrRsVFRVs2rSJzz77jBMnTtC1a1fGjh3LBx98QEVFBU899RQDBw6kV69ejVaO9u3bM3ToUJ544gk+++wzHBwcmDp1qkmN99Zbb6Vv376MGjWKd955h3bt2pGUlMS6deu46667LCqPk5MTM2bM4MUXX0Sj0dC/f3/S09M5fvw4EydOZOzYscybN4+RI0caOzYvXLjAypUrefHFF2nRokWj3fOVIgO0nUvOLeZkSj7n0wvJLS636ti9m1YTe3g3+/5YbXGANqdcJ0jMKSYxp5idZzPxdnGkTYA7rQNcCfJwMnbQSLbVqlUrDhw4wOzZs3nuuedITk7G39+fqKgoPvvsMxRFYfXq1Tz99NPcdNNNqFQqhg4dyscff9zoZfn222957LHHGDhwIIGBgbz11lu8+uqrxu2KorB+/XpeeeUVHnnkEdLT0wkKCuKmm24yjsKwxKuvvoqDgwOvvfYaSUlJBAcH8+9//xsAFxcXtm3bxowZM7j77rvJz88nNDSUwYMHN5satUwae4Xt27ePF198kXfeecdYqyit0HEyOZ8jiblk5Jdadb6s1EQKc7NRFIUvX3mMgpws3Lx8eXz2AoQQuHp64xMY2mjld3dyoEuoJ11CPXHT2v/nvUwaKzWFxkoaa/9/Uc3c999/z5YtW1i0aBFh7bpwOCGH06n5lOsa9rn51rhbarxWkJPJe5PvNj5/7/dTDS5vdfklFfx9NpPd57JoHeBKt1AvwnycZa1akhqBDNBXwIULF8jIyEBRFH788UcAvlu0BIf2gy67ljt2xjx+ePcl9Dqd2e1DH36mweWui14IzqQWcCa1AG8XR64L96ZziAcOajkXSpIaSgboK8C0V9tQ08zNbpxabtTgOwkMb21yrqoK87IbdF5rZBeV8+fJNPbGZREV4U3XUE8ZqCWpAeRfzRUwf8E3qNWVn42mTRkqtZqxM+bZ7NoHo9dz8cxxEk4fIys10WbXAUPzR/SpdL7ZcZ79F7Ip1zXuWFtJau5kgG5CxWU6Np1IpSyyP89+9JPZfaZ+tJyowXde1nXcvHxx9/Yzu62yPfr9KfeYba+uLuH0UT594WESTh9tcHkKS3VsO53ON9vPc+RijkWz4mzNHsogNV+N9fslA3QT0OsFhxJyWLgzjmOJuSbjlys70xqzU83LP4hXF23hwRnzUKnVZvextKZedaje5Soq07E5Jo3Fu+NJyCq67PM1hPrS+9FcFtOR7FNRkeH3+3LX0LZpG/S2bduYN28e+/fvJzk5mVWrVjFq1ChbXtLuJOYUs+VkGunVhstV1nK9/IPpM/Redm9YQU56Mm5evjXOUd+EE3PbHTQaeg2+k6Ba2qOnfrQcIfR8+sLDNc5bdajeoa3rAEPTyPW33dUoQ/Uy8ktZsf8irQPcuKmtH14umgafy1oODg64uLiQnp6Oo6MjKpWso0iNRwhBUVERaWlpeHl5GSsEDWXTAF1YWEj37t159NFHuftu851WzVVZhZ7tsekcuZhrdsZfZS1X7eiIoij0HT4GXXk5DpqawaqyFrtl+TcU5GTWCKjVa7nmgrmiKAghjD+rH1d136Yaqnc2rYC4jEKiIrzp09KnSToSFUUhODiY8+fPc+HCBZtfT7o2eXl51boaoDVsGqCHDRvGsGHDbHkJu3Qxu4hNJ1LJKap75l/VYKwoislzc7XY43//SXlZCVtWfEO76/qxc+0ybrr7YZNabn52JrGHd7N9zVIeeH5OjZr6jl+XkpuRSn52Zq2147qG6qnUah54fm6N1xtKpxfsOZ/F6dR8bu0YSJiPS6OduzYajYa2bdvKZg7JJhwdHS+75lypyWYSKopSbxNHaWkppaX/NAXk5eURFhZ21cwkrNDp2XE2k4Px2Ze9aNH029tfdnmmz1+JEAKtiyv+oZEoimLRead9soLlH87i4pljZs/Zom3nyy5bbTqHeHBTO3+cHBvnF1yS7I01MwntKkDPmjWL119/vcbrV0OATs0rYePxFDILGqdWtn/zmjonnNTNBQg0eYx68k1UasHFM4fZ+/tyhCgHyoEU4AKQgEqt44Hn53Lh5BG2r14E1GwaCWvXhXufmXVZa3vUx1WrZlD7ANoFute/syRdZa7aAH211qAPxGez/UwGOn3jvpUXzxyvdcKJgRboAvSs8ugINCywObkW4OVfRkbiZirKtwO7CQjLp2v/GzmxO5qMpAuUl5Zw46hx3PXUfxp0DWu0C3RncMcAWZuWmpWrdi0OrVZrTI9zNSgpN4xrjk0rMLu9Ict91k0BegMjgGFAV6C2YTzFQCpBkR74Bmtw1OgReoWC3DwSz56mpLAYQ4APBcIBF0oK3UgpBBh96QFpCSVsXnYAaIfWZR+wqlFHdNTldGo+STnF3N45kAjfmnn4JKm5s6sAfTVJyyth7ZHkOpcAtXa5z+oB3c3LFzevcNQOw8nN6AUMx9BkUVUGcIDgyEKS434AjgBJQAEgePAF0zbjlfPf4uyRRThqnQiKaEufofey67cVpMZnU14agCFYdwb6ADcA/kA/oB+lRQAFFORs4L3JK4B1QG6jLr5UXUFpBasOJtI9zIsb2/jJKePSNcWmAbqgoIDY2Fjj8/Pnz3Po0CF8fHwIDw+35aVt6nBCDttOp1NhpknjcsYQVw3oaQme/PpVHqXFsZSX/lNLVjsWoSv/FfgV2AYkAHDHY1+y7H9b8PQL5IZhz5uMqzZXJo2TC/c+MwuA9r0GEL38a3b8uhQ4APxSpVStUJR+CNEXuANDAL/30qOM4MhEDmxxolv/Ahw0tmktEwIOxeeQkFXE0M5BBHjIZUKla4NN26Cjo6O5+eaba7w+fvx4Fi5cWO/x9rYedIVOzx8xacQk59W6jyWjJKrWOKsGzy/+7xkKc29HpZ6MXtfTuI9PYDmd+uTSuV8xfsFxfDT17kuB+D5jIJ72yc+4efoYx1ULIYzjqi0pk5uXDwU5WYYnimKIipd+jp0xjyVvv3BpzyhgFHAXhpr2peM9K+jYO5a0hJe5e8pDtX5juNxmH7VK4ca2flwX7m31sZJkD+yyk7Ah7ClAF5RWsPZwEsm5JXXuV9foi8oxxFXX2jAETz/geeDfgOelLWXASuALpn3yDPBP7buirMxsIG5Imerj7u3H/c/NYcF/JtUY0fHIa7+TeK4nu37zJC+z8suYHu/Ao9wzxY+OvQupPoN95fy32L560WV3NLYJcOO2ToGyA1G66sgA3chSckv49XASBaUVFu1f2+iL6mOIC3LULPtfJid2dwYqO8FigS+BhUB6jXM0tL23tjKpVGr0evMfJvc/N4ceNw2jIDeL96fcYzItPTM5noCwVgy+/3HcPAM5dyyY9d8Wo6v45xuTozaGG0fF0/dfGoryGz/Li6ezI//qGkyQp2zykK4eMkA3opjkPP44kWq2vbk2lcGweo2zMkAX5KjZssKbHau9KCut7PTaB7yOoeOt5rXM1b6tUVuZTJsv/lH9w6R6rX3lJ29carOurjWGbwJP8M9wvyPAm8DPZu8NGv7Bo1YpDGjrR0/Z5CFdJayJa7JLvBZCCP46k86GYylWBWf4ZyGkFm27cO8zr9OibRfcvf1wcvVj84/evPVwS7b85ENZqYqwdiWMenIncD2KYj44w+UvQ1pbmVzcvYD6V9Vz0GjITksi4fQxEmNPcPivDQA4ap2r7XkWeAGIxBCUc4FuwHLgKHCPyd6WrqpX27KnOr1g66l0fj2cRGlFQyb1SJL9ksPszCjX6dlwLKXW8c31yc9OJ6BFS0Y8/iLh7bvRd/gYTuzSsuA/IaRfNLQVh7UrYci4TDr2LiQ3o4LNywzrZbTr2Y/Ny74wnqvq4kaXo7bFmQpysyxeVc/cIkrlpcW1XDELeA14D3gGmIqhU3EFsBWYBhysc1W9quobshibVkBWYRkjuofg49p0q+NJki3JAF1NUVkFaw7V3xlYl72bVnP26F72b16Dq0dPVn8ewLG/3QBw965gxKR0ogbnGzvQqgbP3IxU9mz8ucYoDXMB01rmFmeyZlW9+vIdmpcDvAF8ADyHoTN0IIYmnW8pzNVyfPcys8HX2iGLWYVl/LAnniGdA2kTIKeJS1c/2QZdRVZhGb8cTKxz8kmtx1YJJoaOsDw0Tq+j081AV65GUem56a4chjyUhZNr3amfrB2l0ZTqn35eO0etE34hA3DQvEfCKUMgdtSUoajepqzkLdy83E06Di3J+DLtkxVmh+31ivSmf2s/VCqZXVyyL1ftVO8rKSGriLVHkikpb1g7pmkw6QhspKyk16XnmxH6Zxj5xCqLzlXXMqT2TlGpEHrzH0AaJxfuf+5J4AzvT3kc+IDysj7Aq8A9FOQ8ZhL8h45/hg3ffWT2XJWdprU1feyLyyYtr5R/dQ3GWSOH4klXJ9lJiGGkxqqDibUGZ0vy8g0d/wyGt3M6htl4vYBs4CEU1RDGznik8Qt+BVR2NgaGtzG7/bE3Pq81H2LhpeF670+5B9iFohoAPASkAp2A7cDHgDvtowZQkFN7BnL/0EgEwqTpo3oy3PisIn7YE18jm40kXS2u+SaOfXFZbI/NqHP9ZksmVyx95wv2/TESuOnSK+uBx4Bkm6+h3NQqyspIjjvN+1PuMTuUMCiiLYf+2sCyeibsCARL334R8AbmARMv7XERjdPzqB02UFyQa3ip2uzG+lQdtqdxUHF7p0DayuVLJTsgmzgsIIQg+nQ6h+JzzG63pIMKoDA3mxO7I9j3x+uAG5CPoTNswaVg0hR307QcNBrcvf1qHf1hST7EFm078/3saZdeycbwYbYU+AJoQ1nJMmAZMBnI+ico1xOczWV8KavQs/ZIMn1altK3tW+jJuiVJFu6JgN0hU7PxuOpnE7Nr3UfS/LygRPwCVD5WjTwCBBneCoE7t5+NUZgNP4ypHXTOKhwUCnoBQgEQhgyjesF6Bv4Bcqa0R/Va9mp8WcBOH1wp3E7gBB/YhgzPRPDh9z9GL6RPAL8blG5KoO/ObvPZ5FeUMrQLkFoHWS7tGT/rrkAXVKu49fDSVzMrm38rkF9efmGP/o50SuGkZ8dCugwBJX/UlllrjpVunrQsnYZ0vo4qBR83bQEemjxddPiqlHjonUw/NQ4oHGovauhqKyC/JIK8kvKySsx/DunqIyU3BKKyuruMK2vM9Nc5vKE00drzFw0bWUrBl4CfgIWU9nhCh+hKP+HEIW1lEbBkq8r59IL+XFvAnd2D2nSbOJS8xOfWYSXqyMeTrWtyX75rqk26PyScn45mEiGhWmpahtSdsfEXWz6oRelRWoMHVwPAFtM9qne7nx8159sXDSfm+5+mDVfzL2s9ShctWpa+rkR7OlEgIcWX1ctahsMJ8stKic5r5jk3BJScktIyyu1usZdfcjg3t9X8dMH/6ljLHXVQOsMvANMAcDDJ5O8rNuAgyZH3Hr/E5w6sNO4qp+Xf/3ZlJ0c1dzRLbhJktRKzU9GQSk/7UtgbO8IPF2sC9ByLQ4zMgtKWXUwkfwSyxY8gprrVxiyl8wDngVApd6BXjcaSK7RiVU9QFu7DGl1fu5aWvu50srfjUAP7RVpRy0u03E2vYCz6QXEZxZZPQW+Um0ffC4eXvgGhdG1/21s+P4jQDBs/FR2bywkM2kWQgRiyKP4KobAbbj+9PkrCW3Tyerx4ipFYWB7f3qEeTXoPqRrU0FpBcv2xJNfUsGj/VvaNEBfE00ciTnFrDmUxJnjh6xq+636Ff26QQ+xcdFASov7XNo6B73uVQzNG5h0YlW2O1ftaNS6uFJaZP7rubmOLQAPZ0e6hHjQIdgDT2fbfY2ylLNGTZdQT7qEelJWoedCZiFn0go4m1bQoGBdvW36sTe+IKJjdxRFYeDdEwBw1Gq5ZYwgLyOLHz9QcXKvPzCXoIiJqBwmkp8Vg5uXb4PGi+uFYMvJNDILSrm5fYCc1CLVq6xCz5pDSVZV9C5Hsw/QsWn5/HbUsOCRtW2/lR1hmSkufP1aC0qLNTg4lqLT3Y/Q/1Jj/+rtzpbUmsG0Y0ulKLT0d6VrqCeRvi52O+JA46CibaA7bQPdKSqr4ERSHkcTc8kpqn8Wprm26Zz0ZLz8g4z361glN6WiKHj6q5n0VjY7fy1j9ZfBpFxoi6ffZh56KQEvf+v/WKp21EJXsgrLuKNbiJzUItVKrxf8diyZ1LyGLwNhrWYdoA8n5PDztoMU5DQsBRXA2SNefDc7mJJCNd6B5Ux8Ixl9xcO8N/mXGvtWH0FQ/9oV/7S3Ojmq6RHmRZdQD9xt2OlgCy4aB3pF+hAV4c2FzCKOJOZyPr2w1vZqa0aAVKUo0P/OQlp2iee7twwLT33xckuGTcjg5vuyUVkx7arqhzXApwvmcfTpl3l2zFB83a6exMVS04k+nca59No6qW2j2Qbo/Rey2HY6gzcfqn+4nLm2XyFg+xovVn/mj16v0LJzMRNeSyIn/RDLP5wF1PyKXl3U4DsJrGUscGXHVm5GMoO6t+H2PpFX/dAvRVGI9HMl0s+VnKIy9pzPIiY532ygvpzp7CGtypj2yQVWfBTIgT89WPeNP2ePujB2RjKuHrWvc1Lb2Pb87ExiD+9mw0/fE9iqM8O6BNHK382KO5eau/0XsjickNvk1222U70TcwxfQ8bOmIdKXXvgG/rwMzVe0+tg5fwAVs0PQK9XuP62XJ58+yLu3jr2blrNxTPHcNQ61Vhbua4V56o3VfS6eRg/rf+ThPgLjOjf9aoPztV5uWi4vXMQE/pF0iXUs9FHmTi5CMbOSOG+aSk4aPSc3OvKe5MjuHim9trvW+Nu4f0p9/De5LuN+RcLcjI5tHU9AHt/X8m5mKPMX/47v24/3Kjlla5exxJz2XY6o8brCaePMmLY7ezbt89m1262NehKddViAQrzTNd7KC9TWDw3iKPb3VEUwfCJGXS/6RjJcXVnxfb0CTBbC6ze3rpn4woKMlN57PYetI40v2ZFc+Lp4shtnQLp08qHveezOJaY1+DJMdUpCtwwLI/wDiUsfD2EjCQNH00N495n0ug9pGZiX0uWS638PXkP2HAsmcEdAnBQN9t6jFSPUyn5/BGTanbb3k2r2b4tmkWLFtGrVy+z+1yuZjvMbs3hJM5eWnC/riUyq45FVjn4s+qTKM4dc0HtqOehGSl0v6ngsofIVY4Fbh3gxsC2/jg7CLTaa7OdM7OglG1n0onLKGrU8xYXqFjydhAndhuaJvoOz+GuJ9Nx0Jj+eluyXGrV9GIhXk7c0S0EV22zr8tI1ZxNL2Dt4WSTCkXNZYWzCAgI4LfffkMIgZ+fHxEREXWeV46DxjRA56Sn8P6Ue8jPrvk15R+hwAagC04uOh59PYk23Q2zDa3N1F2dl4sjg9oH0NLPtdZ9rjXnMwr560w6mRZOGrKEXg9/LPVh4yJfhFAI71DMhFeTTUZ51BzbXlP1MezuTg6M6B5CoIdMTnutSMgq4peDiTWGj5qrrFX/XaovpMqchNVUjhp4sNb26I7A30AXPHwrmPJegjE4g6GZZOpHy82eu65cgQ4qhf5t/Bh3Q4QMztW09HPloT4RDGrvj5Nj47S/q1Rw+0NZPPZmIs7uOuJPOvPe5HDOHfsnsFbNzTj4/idMjq/sJ1j+4WsmS8vml1SwfF8CJ1NqNptIzU9ybjFrDieZHdtvrk+rMiA7ODiwePHiRi3LNRGgAeMKazUD7Q0Y1iEOQ1Gd5r5n/0BXcYBjf/9pdg3o+pKrVgr0cOLBPuH0bukj2zBroVIpXBfuzfh+EXQIarylQDv2LmL6J/GEtCqhIMeBz14M4+91nsA/H9ZTP15O/xEPXgrWnY2dvY5aJxJOHzMOv6tUrhP8djSF7WcyGiVHpGSffo/ewaCbb+HsCfOdxHVV1nbv3s3YsWMbtTzXeMPaIOBXDMuE/o3Q38FXr2WZ7FE5qaW2yRXVR26oVQq9W/rQO9JHzkyzkIvGgWFdg+kQ7MHmmNQGz9Kqvkrg0+8n8ON7QRza6s7yDwNJPKtl1JNpxs7cymCdm5VGUV4OYe268OUrkygvLal1rPzeuCwyLq2I11g1f8k+pOSWMOejLzl9cBeBFkxmq2zaUKlU6GvJInS5rrkAXRloBcMoyP4Cw4I8vwN3AYaOK0WlQuPkQmlRgckf6uT/LcY/NLLWyRV+bhqGdA4iQLZVNkhLP1ce7hvJjrMZHE7IsWRdfhPVZ4pqnQWD7v2d+FOpZKU+yc61XqTEaRj/ajLu3ob+BAeNhtkPD65xrrrGyp/PMKyId0e3YDmppRm4cOECJ85fZOupDPZtWQvUPZmtamVt+pR/s3TRQhISEggICGj0sl0TnYTVHdzixNJ5YegqFGA1MAawLC2SudEaigI9w73p19pXNmc0kuTcYn4/nkpWYd2diOZ61auOzNm+ejF7N62i8w3vcvbIVEqK1Hj5l/Po60m0aGP4P29oJ7DGQSUziDcDliynUP3vvnJk1sQBrfBwdqCsrMzikVl210k4f/58IiMjcXJyok+fPuzZs6cpLmvW3k3uLH7bEJzbRyUA96Io9Y8kUKnVjJ0xr8brWkcVd3QL4aZ2/jI4N6JgT2ce7BNe70pztU0+eW/y3bw/5R72bjIk6r1wcg73P/cn3gH55KQ78vG0MA5tMwzJa2gncGWmlp2xsl36ahWXUcjDL79b62S22v7uHTQazh51YcEXKsNiaDYaNmvziPLjjz8yffp0Zs6cyYEDB+jevTtDhgwhLS3N1peuYedaT36YF4zQK/QZmsvoqedx9/YiKLIdg+9/otZEqGD+DzXAQ8vY3hG0CZDTgm3BUa3i5g4B3HVdKG61jEOub6ZopYKcTBa+eTvZaWHAb5SXqvj+rRA2fO9L1eZDSzuBKwlhyNTyy6Hakw5L9mffvn30vXEg83/aSI+bR1j9AX0hxomvXw3lhelqfvrJduW0eYB+7733mDRpEo888gidOnXi888/x8XFhW+++cZm19y3bx//mTTaZATG9jWerPgoEIAbR2UzemoqPoGGTqLk86fYvOwLUuNjLb5G11BPxvQKs3otWMl6kX6uPHRDBG0Da34Q1lX7NUelLuCBF04w8B5Dbfv3xb4smh2MxsnfOPzO0un7VcVlyAziV5P3P1vAru3b2L3pF5PXLfmATjyr4ctXQiktVnHTQD0jRtiunDbtJCwrK2P//v28/PLLxtdUKhW33norf//9t82u+/3333N07068QlsT1q4r29d4svITQ3C++b4s7piYQeX776DR1DsF2MnV3fiH6qhWuKVDIJ1CbJNlXDLPWaPmjm4hHE/KJfpUOmUVDes1/2fFwQyCIspY8VEgh/9yJyO5F0+/tx3fEGp0AluaQzKnqJwf98YzqH0AXUI9G3inkq1cuHCBjIwMjiXmsmblCuCfzsC8rIxLnYEt6hyllRrvyBcvtaC4QE1kp2KW/uSAs7Pt6rk2DdAZGRnodDoCAwNNXg8MDOTkyZM19i8tLaW09J8aSF6e5RMDKt98RVH48ccfAcOb76iZzp8/tQPgljFZDH/0n+Bcqa71OqZ98jPBke1w0Ghw0zpwZw85o+xK6hziSbCnM+uOJBlTl7l5+eLk4kZJkflOYQPD0q7LP3yNe5+ZRVi7rvQZmod/izIWvh5CYqwTH09vzSMzk4jsVMLFM8eMQdmadcTLdYJNJ1K5mF3M4I4BOMp+CbsRGRlZ47Xqo3Xe+OnvWj+gb7prJj9/cisFuQ6Etilh0luJuLnVPa37ctnVb8+cOXPw9PQ0PsLCwiw+NjIykl69ehEVFUV6ejoABTn38+dPPS7tMddscK6u+lecyqUw/dy1jOkdJoOzHfBx1XB/73A6X/oW4+UfxBs//c20T342u/+t9z9BWDvzk1BadSlh6qVJLfnZDsx/oQV7N7nz1+olxB7eTfSKb0yWJr145jgJp4+RlZpYZxljkvNYtie+3lEoUtMoq9Az9a2P6u0MrP53D5XDNxNY+k5PcjMcCYwo5Yk5F3F2s83YZ5Ny2fLkfn5+qNVqUlNNV4NKTU0lKKhmYs+XX36Z3Nxc4yMhIcHiay1evBgHB8MXAkOP+mTgk0tb5zJkXHKdwbnqFODqbZCt/F0Z0yvMptl7Jes4qlXc3jmIoV2C0DiocNBoam0/DIxow73PzELjZEgQWz3Q+gRW8PT7CbTvlY6uXMUP84LZt+kGQMXB6PVmR4e8Na7mOuPVZRSU8cOeeDlF/AorLK1gxf6LhPceYnFnYFZqIgmnj3HxzHEObtkF/EFJUSCefgXcOWkzZSWWx6bLYdMmDo1GQ1RUFJs3b2bUqFEA6PV6Nm/ezJQpU2rsr9VqGzxcZezYsXTs2JGoqCiqB2d4maL8cXUeX1uWj+vbBDCwnb/dpp661nUM9iDQw4l1R5LIMTPbM+H0UZa8/YLJMeYmoWidBaf2BQKvY0hK+yLQCXgQyDc5vrYckpWqt1n/djSFuIwibu7g3+zW/bZ3KbklrD1SM4dgfck2/vkA9gc2Y1ivJ57cjBtZ8J94oO4VLBuLzZs4pk+fzoIFC/juu++IiYnhySefpLCwkEceecRGV9QAj1/6tyE4g2VfT6vWwlQqhdu7tWBQ+wAZnO1cZZPHDV3bGtfZ6HfH/Uz9eDn3PzfH4jGuY2e8g0r9OvAAUAzcgWERrVYmx9U1NhpqptMCQ5PH0t3xJOcW13qc1LiOJeayfF+CSXCu65tyVWNnzENRBWEIzl2BROBWIL7WsdG20CQzCT/55BPmzZtHSkoKPXr04KOPPqJPnz71HmftTMKLFy9y/fXXk5Kiw/BH9lGt+9b16adWKQztEkS7QDlD7GpzID6bv05nmKzhW9sa0NWXFTXdtxeGWaYhQCYwGkWJRghh9rj6ZjRWThdWKQo3tPKhd0sf+cFvIzq9IPpUGkcumk9RVTkLsLL2bC4fZn62mo+m+pOZ7IEhOA8CDMNwq/7/P9q/pdVDba2Ja02yFseUKVPMNmk0thYtWhAXF8fy5cuZMOERzI2aq+/rqaNa4Y5uIUTK5UGvSj3DvQlw17L+aDKFpaa/APV9rTXddz9C9AJ+AXqjKJvw9Hub0uJ3WDn/Te566hWTER3m2qTNNaXohWDn2UwuZBUxpHMQns6yX6MxFZRWsO5IEkk5tWferp4PMznulEmTVH62ms9ebEFmshZDcL4ZiLXo96ax2dUojsag1Wp56KGHmLd4rdntdX091TqquLtnCxmcr3ItvF14sE8EoV7OgOVfa2vu+29CWz2Jg+NPCKEmJ/3/8PBZRdwJw0iQhNNHjUvS1jWj0dxX4sTsYhbvunBpUSg5TbwxxGcWsXT3hTqDszlVm6Tys9V8+mILUi5ocfcuxdXjbsLaOTVo8lJjaLaLJb2/bCPTHxhao9Zk7uspgKtWzV3XtcDfXa5O1lzo9IKtp9M4nJBr0dfaStX3Tb94gR2/BrN9dReEUIDtuHhMot11rTi0dT3X3343Dzw/x6qmlKpaeDtzW6dAvFwsz2wu/aNCp2fH2UwOxmdbvAKiuSYpF4/OOLvsJDPFA3fvEqa8l4y3f2GdvzfNoonjSvD0sWz9ZgAPZ0fu6Rkq/0CaGbXKMOvT382JLafS0F3KkFF1jKs51b8Cz5045NKzIcAyYABFeRs5tPUuwJAN/MaRD5ESf9Z4jKVNKQAXL9Wm+7b2pWe4t2ybtkJ6fikbjqeQYeUU+5pNUi0pyltNUZ4HcJH87EH4h67HMOjAoL7fG1totjXoNYeTOHUxq95ak4ezI6N7tZBjnJu5xJxi1h1JqtEubQnT5UjbYeg87ACUAE8A35vsb5il+E+lYNonP+Pl/8+4/7qmjgd7OjG4Y6D8JlcPIQQH4nPYGZthNjWVOVXf97SE81X+TzsBmzB0CJ9FUQ3hwRf+XedInUq2rkE3uzboqqpPXjAXnO+NksH5WhDq5cwDvcMbNBPUdEGm0xjSpP0KOAHfAR8Dht+httf1Mxnm9+qiLSbBGcwPw6uUnFvC0t3xbDmZJlfHq0V2YRk/H0hk2+l0i4MzmL7v//yf9gK2YQjOR4EBTPt4HlGD7zTpY7hSmm0TR33cnRy4t2cL2Yt+DXF3Mnxb2hyTSkxyfv0HmGH4NpYLjAReA2YBU4DuwGiSz58iMfZEjUwcVds8q04dN5e1Qy8EhxJyOJWaT7/WvnQN9ZTNHhjamvfEZbE/LtviwFzX+374rzLgT8Ad2A38C/gn5Z01a7DYyjUZoN2dHLg3qoVcKvQa5KhWMbSLIVXVjtgMizuVqqY5atezH5uXfYFh1uF+YDFwI3CAgpx7zKbKsnQYXlXFZTo2x6RxNDGXQe0DjKNSrkXnMwrZcjKN3OJyq46r/X3/FvgJcELrvJuh47dw4M9QslIEedkZXDxz3OwHaW5mGttWLqx3ZcPG0qzboM2lvHLTGoKzt6vsELzWnU0vYMOxFIuXLq0c3ZGbkcr7U+7BUaMlOz0ZoW+NYbx0J6AMeB5F9SkPvjCXgLCW/LpgHm169OH3xfOtTqtVVSt/V/q28r2mcl7mFpez9XR6renr6mM+ndmTGJql1IS2Ps3TH4BGa2jXfm5IB4vOe+Oocdz11H9kG3RjctGouUcGZ+mS1v5ujLk+zOJmrso+jcp1W175fjPTPl4BnAH6ACsw9Pp/RNvucXS+YZTxa3JhbnaD0mpVdS69kKV74vn1cBIZBc07MUBBaQV/nkzlu51xNYKzNW3Dpv0HCjAP+BRQ07lvHFM/NgRnMDRf1TWeXVGp0LoYkkYcjF5PUdIZTh47xIULFxp2kxa4Zpo4NA4q7rouFB8ZnKUq/Ny0PNA7nF+PJJGYbfk6GdU7nBWlECFGA88C8zh9sAXvPF5AaUkcYPiDDm/frXJnsGIYXlVCQGxaAWfTC2gX6E6flj7NKrN4YWkFe+OyOHoxt9Z25oa1DTsDi4B7Lj3/P24fez1qB9Px6XWtDS/0ekovrTlekJPFfybcwX8qt9moIeKaCNBqlcKIbiHX1FdDyXLOGjX39GzBlpOG9l5ruJldQe9G4Cdy0sOBjcAUCnK++WdVPSG495nX6xybXx8h4FRKPqdS8gn3caF7mBet/V2v2s7E/JJyDsbncORiDuW6msHOmk7W6hQlCJV6G3pdL1SqCrwCXqG89BvcvYfWWaa6P0ANrzs4OLBw4UKL79Nazb4NWlFgWJdg2gfJhY+k+h2Mz2ZbtcWW6lN95uHe31fx4/vvIfQLMYwMAENH4lMoqkIeeH4uvW4dWe+MRmt5ODvSvYUnXUI9cXK8ssua7tu3jxdffJF33nmHXr16md1HCMGFzCKOJOZyPr2wzvd8+u3t672muQXQUi5o+Pq1EDKTNTi76Zj4ehItuxTV+b7npKfw/pR7TD50M5MTKMrPqbHv/v376dmzZ71lq0rOJKxiYDt/GZwli10X7o2Pq4Z1R5MpLbes87D6zMPeQ+4mpFV73pt8BzADeAt4COjHmGkJ9Lo12LhvYwTnqhMw8oq7sutcJq383Wgb4Eakn+sVSbv1/fffs2XLFhYtWlQjQBeVVXA8KY+jF3MtHpVRV97Q2hZAO7zNjR/eDaKsRIVvcBmT3kokIKwcqPt9N7c2/IWYw3w0dQwqlQq9Xm/8aWvNOkD3bunDdeHeV7oY0lUmwteVB64PZ83hpMtMWSVQlLcRYhuGGnQrfnw/kuy0LG59MIta+qKsVr1NtlwnjM0fGgcVLf1cmyRYm8sLumzZMsaPH09ecRkFuFCs9SYpp8SqbyhQd9vwP4mADXQ6WP+NH1uW+wDg7LaPe59JJiCs/lp4peofuv27t+OnoCDCwsKYOHEiX3/9NQkJCQQEBFh1H9ZqtgG6U7AHbQLcrnQxpKuUt6uGMdeH8duxZOIyiqw+3rRtegh/rxtLasI0KspGs3GRH6f2uzJ2RjK+wRX1n8wMS9pkC3OzjDXrU+264qhWCPRwItTLmRAvZ4K9nBo1w0vVpKyVbeFpaemXshwZNDQLScLpoyz/cJbx3ObWOkk4fZRfPltARfn3JJw2BOfQNutIjB3Jid0P0j7qP+ZOXa/eLX3o36YdI+Pi0FwayfP4449TVlbW4AxQlmq2AVoGZ+lyOTmqGdk9lL9iMzhwIduqY6t/TW7RthO/LphH2+siiF4eRdwJZ959MoKRT6TTZ2hevcmMq7Nk4suAkeNq1KwvZhdz8dJoFZWi4OeuIdDdCU8XRzydDQ8PJ0ecNfUHbr1eUFyuo6hMR15JOTP/9xlvvfg0Ol1FlcBp+FnfOuz12btpNRfPHMNR60RQRFuzC6D9+dMRzh//EgjAUVPBkHH7if75YUBncYdidTe18yMqwhDsqwZjRVFsHpyhGXcSSlJjOp6Uy58xaVat/VDVyvlvsX31Im4cNY6Bd89iydvBnD9umBnY7rpC7puWik+Q5bVp8xMwDBSViqHjnuav1YtqzexSH42DCq2DCrVKQa1SUCmVP6GkXE9RmY7SCl2NmZgNXXLVHHNLgrp6+vD47AUAuHh4oa+ooKiggP2b27F9dTsM49BPAXcBMbWeu76avKLArR0D6RLqaVWZLWFNXJMBWpIslJxbzK+HLV8Rr640WDqd4PiubkSvaEVFmQqNk57hEzPoPyIHlYXNxLUFw/rYMtlpZZksXYe9LpaM3IBwDKsJDrz0/GfgEaon+q1kyaxNW6e8k6M4JMkGgj0NK+L9ejiZ1Lz6s3ZY0gzx8rfn+PG9QM4ddWHV/AAObXVjzPRUAlpYvuZE9WCoUqnR6y0f7dCYzI0Lb+hY77pGbigqNX3/tZo9G2+hotwZQ0B+BlhY5zmrdyhW56hWGN4thJZ2klVJ1qAlyUoVOj1/WLAiXl3NEFVrcno9/L3Wk7Vf+1NarELtqOfme7MZfH8WWmfTTrCq60ibG6+bk57M/c/NYcF/JtW4ZkNqsQ1hTfaa+pj/luBN+6gjnNrf4tLzHcA44HyN462pyTs5qhnZI4QQGy9KJWvQkmRDDpdWxAvwcKqRQbwqS4eGqVTQ/85cvAIO8MM8L4ry+/LHD77s2+TBiMfT6THQMOGq+nA6c+N1deXlpFw4A9Q92qG2hAGN8v5oGj8LyT/3MB54h1P7A1Cp9Dho5lJW8hqgM06hr/ypUjvgHxrBjaMerrcm7+7kwF3XhdrdtHkZoCWpgXqGe+PvZsggXlRWd7u0JcHy1P7VFOUvomPv90m9MJmsVEcW/TeELT+lc/N9R+qd4lwZDOtrZrCHdY4tVXkvLh43oit/n4ykiEtbTqDXj6esZN8/O1e+r0LgqHWivLSEttf1pd8d9xs/vMx9WPi5aRh1XSjudpi4QzZxSNJlyi8pZ+2RZFJya7ZL19YMUZkGa+m8l9i3aRXXDfoXZw7tMnYkPjLza/b90Za/17XEsNCPDvgaw6zEBJNrmOv0q97McGT7JjYv+4Kb7n6YNV/MbfDoDlurXrsvKVLY8J0321f7otcraLR6OvbeypEd/0Loa77fikrFkIem8NfqxRTm1n+PoV7O3NkjpEmnxstRHJLUxCp0eracSueYmcWWqgfLjMQLlBQVoCiKBaMwwoH/Afdeel4KLAD+i0qdZtE60tDwtSwaW9UADNRoaqkcjtj/zgmEtJzLxsW+5GUavuh3HZDPqH+n4x1Q0SgjWFr5u/KvrsFNPhVetkFLUhNzUKu4rVMgwZ5ObDlpOl66epvsnEeHmDuFWUMfHkWnPmremzwAeAO4BUOKrYn0GJhIu54Khtp1TVWH+WldXCktKjS7X1OM7qhUtXlFCIg9vJvta5Yw4M6HUBSFg9EbgAnsXDsLoQ8EwN07l/ufy6dj75ozOhs6gqVHuBcD2/qjUtn36n8yQEtSI+oS6kmAh5b1R5LJLjI/VK6u4WPVFeZVzmDcgaLcihADgTeBARz4sw1Hd+iJGpzHjSNzCG5pum6IuWF+5kz9aDlC6Pn0hYdt0nFY9YPiwJY1AOz7Y41x+97fV7H399XAgxhGZLRF6AFSgDnkZ39Bx95HTM5ZWzt7bSNYKjtlVYrCwPb+9AjzatR7tBUZoCWpkQW4O/FAn3A2nUjlTGrNVE11je6o7mD0ejr0uhEXDy88fQPpP+Jmdv32FJnJ7fEO/Jqksx7sWu/FrvVetO5eRMfrDxOz53nufHy6BR8ECpVTsW3ZcWjug6K4oLIpyA+YADwBtLn0WjrwNobMJ8VoXdy4eOa4STtyQ0awaBxU/KtrsN2McbaEbIOWJBs6GJ/NX2cy0FWbIl59xp2l3vv9lHFssdpRw7mjzmxf7cXRHW7o9ZVf1y8Q0XEvD7wQRVnxIbMfBLfe/wTHd0eTm5F6KZDPsFnHofnx4AOAf2NoW68c2paFISXVx4D55pjK96A2tXXK/mfBLzxyey/83a/8MDq76CScPXs269at49ChQ2g0GnJycqw+hwzQUnOQnFvMb0dTTNY+rh5Idq3/ifSkC5SXFJttQwUY+vAz3P7Q5BqvZ6UmkhJXwpG/WrFnkx+If8b6egekkp32IYZ8iWeMr0+fv9KiGnxjdBwmnD7KTx/MIjFWDYwA7geqThbZC3wOLANqXznQ0uS61Ttl/VxU3HN9S1y19tFgYBedhGVlZYwePZq+ffvy9ddf2+oykmT3gj2dGXtDOFtOphOTnAeYXxS+8it6bYHzn/Zo09EQ70+5t8peWmAM8ABwK9lpgcB/gf/i5ZeMTvcbFWV/oHYIbNAi+NYoK1WIPezC2q+8SIlbD4RVvRtgKfAFinLA+C3CzcuXPkPvZfOyL2qcr75p2pWqdsp2a+HFoPb+OFyBpAWNwWYB+vXXXwewab4uSbpaaB3UDO0SREs/VzafTKW0XG/1jLuqE1S2r15ibDM2DbSlGBYP+h7wQVHuJjD8ZdISWpKTEQw8CjzKvCcgKDKKLn3v5sj21zHUrs8BqYDlwRAMHxarP/+EXrfNpLysE0lntVw4qSL9ogu6CjVQ2UxShNZ5Jy3axXHu6ItALv3veJAzh3JJSziLEAIhBEERbSrflAYn13VQKdzcIcAmq9E1JZu3QS9cuJCpU6da1MRRWlpKaek/6eTz8vIICwuTTRxSs5JXUs6GYylms4hXNn3kZ2dYdK7KNuOU+LMsrUxKW0Xl2hMFuSpidrtx7pgz5485k3axtg+DIuAcrbr44hvsgtpR4FDloaiguEBFQY6awlwHCnLVZCQVU17qAZirpSYAa4FfgS1A/YtMVVU1uW7l5J76eDg7MqJbsN0mibaLNuhK1gToWbNmGWveVckALTU3Qgj2Xchm19nMGmtMV5SVceivDSyzcCheVZYuDpSfreb8cWdi9urZt+k80AqdLhhEw5sCFCWJyM4qAlrkoNcfYN8fsxD6MzX2q9p8Utea1g1Jrhvp58KwLsFXPGluXWwWoF966SXefvvtOveJiYmhQ4cOxueyBi1JtcsqLGPTiRSScmrWLK2ZLaeoVGidXfEPjTQ7pbwulZ1qugqF7FQH0i+qyMlwprhQRUW5gq5coaJcISc9i3PHDtMhqjN7fv8cw3C4ysdFwLTWX1tHZNUPjcZa4F+tUujb2pdeEd7GdFv2ymadhM899xwTJkyoc59WrVpZc0oTWq22SdLISJK98HHVcF+vMA7E5/D32QzKdTXrS7UttFTVtI9XEBTRtkanY121zurrXjg4gn+LCvxbAJTV2GfvptXkZS5C6zKOoeO92fDdUvMnVhTGvviOVeW3ZJ/a+LppGNo5yG6bNC6HVQHa398ff39/W5VFkq5JiqIQFeFNa39XNp1INeYMNDdbLjM5gaL8HLMBzdpOR0smp/x1qTMyesU3nDm0CzB0Vrbp3qfW8zq7eRAY3pq8rIxLY6lb1Lp4/+Us8K8o0D3MiwFt/Jp8PY2mYrM26Pj4eLKyslizZg3z5s3jr7/+AqBNmza4uVmW0FWOg5auNUIIjiflsfNsBoWluhpjejOTE/h42gO1ro5Xn7rScFVOTgGM+zRkQaLq/rfxZJ2L9zdkgX83rQO3dw4kwvfqmRVYyS46CSdMmMB3331X4/UtW7YwaNAgi84hA7R0rSop17H7fBaH4nNqJAS4nIwlluX5axyWTiyxhqJAlxBPBrT1s+uOwLrYRYBuDDJAS9e6zIJSok+lE59V+ww7a1iShgtqH11hjcZOsRXgoWVwh0CCPK/utmZr4lrzbLiRpGbC103LPVEtGNE9GF+3y08dFTX4TqZ+tNzstqkfLSdq8J0EhLUkpFVHq89dOXqisUdRaB1V3NwhgAd7h1/1wdla9jE5XZKkOrUJcKe1vxsnU/LZfS6z1qVMrVHbyIm9m1Zz8cwxk30q9b79bvb8vrLGOVw8vPANCrvsTN5VqRSFjsHu9G/jZzfraDS1a/OuJekqpCgKHYM9aB/oTkxKHrvPZZkswGQpcyMnslIukpedwcUzx425DxVFISCsNV36Deb4rj8pyMmi3x0PELN3G55+gdww7D5jMH76/R/wDQ6zeIhf3fcJHYLcuaGVL14ul/+t4Wom26Al6Sql1wtiUvI4lJBDWl5p/QdUUb2j8bkhHeo95p21R3HQaC6rk7IuigLtAg2B2ce1+QZmu1jNTpIk21KpFDqHeNI5xJPEnGIOxecQm1ZQY9SHOdXHTFuysl3lMdaOt66Po1qhXaA7PSO88XOTE9WqkgFakpqBUC9nQr2cyS8p5+jFXE4k55FfUmHx8XVlebFmZTtr+Lpp6BrqScdgj6t2yJytyQAtSc2Iu5Mj/dr40be1L0m5JZxOzSc2tYCCUsuD9eVMu66P1lFFKz9XuoR60sLbpVHP3RzJAC1JzZCiKMZa9aB2/iTmFHMmrYCL2cVkFpRiLu5ezrTruvi4amjp50pLP1dCvZztPpO2PZGdhJJ0jSkp15GUU0xSTglJOcWk5pUYlzy93A5AR7WCr5sWfzctAR5awn1crvmRGNXJTkJJkmrl5Kimlb8brfwNa+IIISgorSCnqJzc4nLyisvJKS6ntEJHhU6g0wt0wvBTrxdoHdU4O6pxclTjolHjrFHj7uSAv5sWbxeNrCE3IhmgJekapygK7k6OuDs5mmQNlK48OdVbkiTJTskALUmSZKdkgJYkSbJTMkBLkiTZKRmgJUmS7JQM0JIkSXZKBmhJkiQ7JQO0JEmSnZIBWpIkyU7JAC1JkmSnZICWJEmyUzJAS5Ik2SkZoCVJkuyUDNCSJEl2SgZoSZIkOyUDtCRJkp2SAVqSJMlO2SxAx8XFMXHiRFq2bImzszOtW7dm5syZlJWV2eqSkiRJzYrNUl6dPHkSvV7PF198QZs2bTh27BiTJk2isLCQd99911aXlSRJajaaNKv3vHnz+Oyzzzh37pxF+8us3pIkNTd2m9U7NzcXHx+fWreXlpZSWlpqfJ6Xl9cUxZIkSbJLTdZJGBsby8cff8wTTzxR6z5z5szB09PT+AgLkzmGJUm6dlkdoF966SUURanzcfLkSZNjEhMTGTp0KKNHj2bSpEm1nvvll18mNzfX+EhISLD+jiRJkpoJq9ug09PTyczMrHOfVq1aodFoAEhKSmLQoEHccMMNLFy4EJXK8s8E2QYtSVJzY9M2aH9/f/z9/S3aNzExkZtvvpmoqCi+/fZbq4KzJEnStc5mnYSJiYkMGjSIiIgI3n33XdLT043bgoKCbHVZSZKkZsNmAXrTpk3ExsYSGxtLixYtTLY14cg+SZKkq5bN2hwmTJiAEMLsQ5IkSaqfbBSWJEmyUzJAS5Ik2SkZoCVJkuyUDNCSJEl2SgZoSZIkOyUDtCRJkp2SAVqSJMlOyQAtSZJkp2SAliRJslMyQEuSJNkpGaAlSZLslAzQkiRJdkoGaEmSJDslA7QkSZKdkgFakiTJTskALUmSZKdkgJYkSbJTMkBLkiTZKRmgJUmS7JQM0JIkSXZKBmhJkiQ7JQO0JEmSnZIBWpIkyU7JAC1JkmSnZICWJEmyUzJAS5Ik2SkZoCVJkuyUTQP0nXfeSXh4OE5OTgQHBzNu3DiSkpJseUlJkqRmw6YB+uabb+ann37i1KlT/Pzzz5w9e5Z7773XlpeUJElqNhQhhGiqi61Zs4ZRo0ZRWlqKo6Njvfvn5eXh6elJbm4uHh4eTVBCSZIk27Imrjk0UZnIyspiyZIl9OvXr9bgXFpaSmlpqfF5bm4uYLghSZKk5qAynllUNxY29uKLLwoXFxcBiBtuuEFkZGTUuu/MmTMFIB/yIR/y0ewfCQkJ9cZPq5s4XnrpJd5+++0694mJiaFDhw4AZGRkkJWVxYULF3j99dfx9PRk7dq1KIpS47jqNWi9Xk9WVha+vr5m969NXl4eYWFhJCQkNNumkeZ+j/L+rn7N/R4ben9CCPLz8wkJCUGlqrsb0OoAnZ6eTmZmZp37tGrVCo1GU+P1ixcvEhYWxs6dO+nbt681l7XKtdB23dzvUd7f1a+532NT3J/VbdD+/v74+/s36GJ6vR7ApJYsSZIkmWezTsLdu3ezd+9eBgwYgLe3N2fPnuXVV1+ldevWNq09S5IkNRc2Gwft4uLCypUrGTx4MO3bt2fixIl069aNrVu3otVqbXVZALRaLTNnzrT5da6k5n6P8v6ufs39Hpvi/pp0HLQkSZJkObkWhyRJkp2SAVqSJMlOyQAtSZJkp2SAliRJslNXbYCeP38+kZGRODk50adPH/bs2VPn/suXL6dDhw44OTnRtWtX1q9f30QlbThr7nHBggXceOONeHt74+3tza233lrve3KlWft/WGnZsmUoisKoUaNsW8DLZO395eTkMHnyZIKDg9FqtbRr187uf0+tvccPPviA9u3b4+zsTFhYGNOmTaOkpKSJSmudbdu2MWLECEJCQlAUhV9++aXeY6Kjo+nZsydarZY2bdqwcOHCyytEoyy40cSWLVsmNBqN+Oabb8Tx48fFpEmThJeXl0hNTTW7/44dO4RarRbvvPOOOHHihPjPf/4jHB0dxdGjR5u45Jaz9h4ffPBBMX/+fHHw4EERExMjJkyYIDw9PcXFixebuOSWsfb+Kp0/f16EhoaKG2+8UYwcObJpCtsA1t5faWmp6NWrl/jXv/4ltm/fLs6fPy+io6PFoUOHmrjklrP2HpcsWSK0Wq1YsmSJOH/+vNi4caMIDg4W06ZNa+KSW2b9+vXilVdeEStXrhSAWLVqVZ37nzt3Tri4uIjp06eLEydOiI8//lio1WqxYcOGBpfhqgzQvXv3FpMnTzY+1+l0IiQkRMyZM8fs/vfdd58YPny4yWt9+vQRTzzxhE3LeTmsvcfqKioqhLu7u/juu+9sVcTL0pD7q6ioEP369RNfffWVGD9+vF0HaGvv77PPPhOtWrUSZWVlTVXEy2btPU6ePFnccsstJq9Nnz5d9O/f36blbAyWBOgXX3xRdO7c2eS1MWPGiCFDhjT4ulddE0dZWRn79+/n1ltvNb6mUqm49dZb+fvvv80e8/fff5vsDzBkyJBa97/SGnKP1RUVFVFeXo6Pj4+titlgDb2/N954g4CAACZOnNgUxWywhtzfmjVr6Nu3L5MnTyYwMJAuXbrw3//+F51O11TFtkpD7rFfv37s37/f2Axy7tw51q9fz7/+9a8mKbOt2SLONNl60I0lIyMDnU5HYGCgyeuBgYGcPHnS7DEpKSlm909JSbFZOS9HQ+6xuhkzZhASElLjF8YeNOT+tm/fztdff82hQ4eaoISXpyH3d+7cOf7880/Gjh3L+vXriY2N5amnnqK8vJyZM2c2RbGt0pB7fPDBB8nIyGDAgAEIIaioqODf//43//d//9cURba52uJMXl4excXFODs7W33Oq64GLdVv7ty5LFu2jFWrVuHk5HSli3PZ8vPzGTduHAsWLMDPz+9KF8cm9Ho9AQEBfPnll0RFRTFmzBheeeUVPv/88ytdtEYTHR3Nf//7Xz799FMOHDjAypUrWbduHW+++eaVLprduupq0H5+fqjValJTU01eT01NJSgoyOwxQUFBVu1/pTXkHiu9++67zJ07lz/++INu3brZspgNZu39nT17lri4OEaMGGF8rXJlRAcHB06dOkXr1q1tW2grNOT/Lzg4GEdHR9RqtfG1jh07kpKSQllZmdnle6+khtzjq6++yrhx43jssccA6Nq1K4WFhTz++OO88sor9a6NbO9qizMeHh4Nqj3DVViD1mg0REVFsXnzZuNrer2ezZs317pKXt++fU32B9i0aZPdrqrXkHsEeOedd3jzzTfZsGEDvXr1aoqiNoi199ehQweOHj3KoUOHjI8777yTm2++mUOHDhEWFtaUxa9XQ/7/+vfvT2xsrPGDB+D06dMEBwfbXXCGht1jUVFRjSBc+YEkmsGSQDaJMw3uXryCli1bJrRarVi4cKE4ceKEePzxx4WXl5dISUkRQggxbtw48dJLLxn337Fjh3BwcBDvvvuuiImJETNnzrwqhtlZc49z584VGo1GrFixQiQnJxsf+fn5V+oW6mTt/VVn76M4rL2/+Ph44e7uLqZMmSJOnTol1q5dKwICAsRbb711pW6hXtbe48yZM4W7u7v44YcfxLlz58Tvv/8uWrduLe67774rdQt1ys/PFwcPHhQHDx4UgHjvvffEwYMHxYULF4QQQrz00kti3Lhxxv0rh9m98MILIiYmRsyfP//aHGYnhBAff/yxCA8PFxqNRvTu3Vvs2rXLuG3gwIFi/PjxJvv/9NNPol27dkKj0YjOnTuLdevWNXGJrWfNPUZERJjNezZz5symL7iFrP0/rMreA7QQ1t/fzp07RZ8+fYRWqxWtWrUSs2fPFhUVFU1cautYc4/l5eVi1qxZonXr1sLJyUmEhYWJp556SmRnZzd9wS2wZcsWs39Tlfc0fvx4MXDgwBrH9OjRQ2g0GtGqVSvx7bffXlYZ5HKjkiRJduqqa4OWJEm6VsgALUmSZKdkgJYkSbJTMkBLkiTZKRmgJUmS7JQM0JIkSXZKBmhJkiQ7JQO0JEmSnZIBWpIkyU7JAC1JkmSnZICWJEmyUzJAS5Ik2an/B1YZg0/EkPxvAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "with torch.no_grad():\n", " # Initialize plot\n", " f, ax = plt.subplots(1, 1, figsize=(4, 3))\n", "\n", " # Get upper and lower confidence bounds\n", " lower, upper = observed_pred.confidence_region(rescale=True)\n", " # Plot training data as black stars\n", " ax.plot(train_x.numpy(), train_y.numpy(), 'k*')\n", " # Plot predictive means as blue line\n", " ax.plot(test_x.numpy(), observed_pred.mean.numpy(), 'b')\n", " # Shade between the lower and upper confidence bounds\n", " ax.fill_between(test_x.numpy(), lower.numpy(), upper.numpy(), alpha=0.5)\n", " ax.set_ylim([-3, 3])\n", " ax.legend(['Observed Data', 'Mean', 'Confidence'])" ] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.18" }, "vscode": { "interpreter": { "hash": "8a61b873b964ebeec997f3785d4891e85789cfba4e32a225b1d76f494e6c6489" } } }, "nbformat": 4, "nbformat_minor": 4 }