{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "# Clustered Multitask QEP (w/ Pyro/QPyTorch High-Level Interface)\n", "\n", "## Introduction\n", "\n", "In this example, we use the Pyro integration for a QEP model with additional latent variables.\n", "\n", "We are modelling a multitask QEP in this example. Rather than assuming a linear correlation among the different tasks, we assume that there is cluster structure for the different tasks. Let's assume there are $k$ different clusters of tasks. The generative model for task $i$ is:\n", "\n", "$$\n", "p(\\mathbf y_i \\mid \\mathbf x_i) = \\int \\sum_{z_i=1}^k p(\\mathbf y_i \\mid \\mathbf f (\\mathbf x_i), z_i) \\: p(z_i) \\: p(\\mathbf f (\\mathbf x_i) ) \\: d \\mathbf f\n", "$$\n", "\n", "where $z_i$ is the cluster assignment for task $i$. There are therefore $k$ latent functions $\\mathbf f = [f_1 \\ldots f_k]$, each modelled by a QEP, representing each cluster.\n", "\n", "Our goal is therefore to infer:\n", "\n", "- The latent functions $f_1 \\ldots f_k$\n", "- The cluster assignments $z_i$ for each task" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import math\n", "import torch\n", "import pyro\n", "import tqdm\n", "import qpytorch\n", "from matplotlib import pyplot as plt\n", "\n", "%matplotlib inline\n", "import matplotlib as mpl\n", "mpl.rc_file_defaults()\n", "\n", "torch.manual_seed(2025)\n", "\n", "# this is for running the notebook in our testing framework\n", "import os\n", "smoke_test = ('CI' in os.environ)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Adding additional latent variables to the likelihood\n", "\n", "The standard QPyTorch variational objects will take care of inferring the latent functions $f_1 \\ldots f_k$. However, we do need to add the additional latent variables $z_i$ to the models. We will do so by creating a custom likelihood that models:\n", "\n", "$$\n", "\\sum_{z_i=1}^k p(\\mathbf y_i \\mid \\mathbf f (\\mathbf x_i), z_i) \\: p(z_i)\n", "$$\n", "\n", "QPyTorch's likelihoods are capable of modeling additional latent variables. Our custom likelihood needs to define the following three functions:\n", "\n", "- `pyro_model` (needs to call through to `super().pyro_model` at the end), which defines the prior distribution for additional latent variables\n", "- `pyro_guide` (needs to call through to `super().pyro_guide` at the end), which defines the variational (guide) distribution for additional latent variables\n", "- `forward`, which defines the observation distributions conditioned on $\\mathbf f (\\mathbf x_i)$ and any additional latent variables." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The pyro_model function\n", "\n", "For each task, we will model the cluster assignment with a `OneHotCategorical` variable, where each cluster has equal probability. The `pyro_model` function will make a `pyro.sample` call to this prior distribution and then call the super method:\n", "\n", "```python\n", " # self.prior_cluster_logits = torch.zeros(num_tasks, num_clusters)\n", "\n", " def pyro_model(self, function_dist, target):\n", " cluster_assignment_samples = pyro.sample(\n", " self.name_prefix + \".cluster_logits\", # self.name_prefix is added by PyroQEP\n", " pyro.distributions.OneHotCategorical(logits=self.prior_cluster_logits).to_event(1)\n", " )\n", " return super().pyro_model(\n", " function_dist,\n", " target,\n", " cluster_assignment_samples=cluster_assignment_samples\n", " )\n", "```\n", "\n", "Note that we are adding an additional argument `cluster_assignment_samples` to the `super().pyro_model` call. This will pass the cluster assignment samples to the `forward` call, which is necessary for inference.\n", "\n", "### The pyro_guide function\n", "\n", "For each task, the variational (guide) diustribution will also be a `OneHotCategorical` variable, which will be defined by the parameter `self.variational_cluster_logits`. The `pyro_guide` function will make a `pyro.sample` call to this prior distribution and then call the super method:\n", "\n", "```python\n", " def pyro_guide(self, function_dist, target):\n", " pyro.sample(\n", " self.name_prefix + \".cluster_logits\", # self.name_prefix is added by PyroQEP\n", " pyro.distributions.OneHotCategorical(logits=self.variational_cluster_logits).to_event(1)\n", " )\n", " return super().pyro_guide(function_dist, target)\n", "```\n", "\n", "Note that we are adding an additional argument `cluster_assignment_samples` to the `super().pyro_model` call. This will pass the cluster assignment samples to the `forward` call, which is necessary for inference.\n", "\n", "\n", "### The forward function\n", "\n", "The `pyro_model` fuction passes the additional keyword argument `cluster_assignment_samples` to the `forward` call. Therefore, our forward method will define the conditional probability $p(\\mathbf y_i \\mid \\mathbf f(\\mathbf x), z_i)$, where $\\mathbf f(\\mathbf x)$ corresponds to the variable `function_samples` and $z_i$ corresponds to the variable `cluster_assignment_samples`.\n", "\n", "In our example $p(\\mathbf y_i \\mid \\mathbf f(\\mathbf x), z_i)$ corresponds to a Gaussian noise model.\n", "\n", "```python\n", " # self.raw_noise is the Gaussian noise parameter\n", " # function_samples is `n x k`\n", " # cluster_assignment_samples is `k x t`, where `t` is the number of tasks\n", "\n", " def forward(self, function_samples, cluster_assignment_samples):\n", " return pyro.distributions.Normal(\n", " loc=(function_samples.unsqueeze(-2) * cluster_assignment_samples).sum(-1),\n", " scale=torch.nn.functional.softplus(self.raw_noise).sqrt()\n", " ).to_event(1)\n", " # The to_event call is necessary because we are returning a multitask distribution,\n", " # where each task dimension corresponds to each of the `t` tasks\n", "```\n", "\n", "This is all we need for inference! However, if we want to use this model to make predictions, the `cluster_assignment_samples` keyword argument will not be passed into the function. Therefore, we need to make sure that `forward` can handle both inference and predictions:\n", "\n", "\n", "```python\n", " def forward(self, function_samples, cluster_assignment_samples=None):\n", " if cluster_assignment_samples is None:\n", " # We'll get here at prediction time\n", " # We'll use the variational distribution when making predictions\n", " cluster_assignment_samples = pyro.sample(\n", " self.name_prefix + \".cluster_logits\", self._cluster_dist(self.variational_cluster_logits)\n", " )\n", " \n", " return pyro.distributions.Normal(\n", " loc=(function_samples.unsqueeze(-2) * cluster_assignment_samples).sum(-1),\n", " scale=torch.nn.functional.softplus(self.raw_noise).sqrt()\n", " ).to_event(1)\n", "```" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "class ClusterGaussianLikelihood(qpytorch.likelihoods.Likelihood):\n", " def __init__(self, num_tasks, num_clusters):\n", " super().__init__()\n", " \n", " # These are parameters/buffers for the cluster assignment latent variables\n", " self.register_buffer(\"prior_cluster_logits\", torch.zeros(num_tasks, num_clusters))\n", " self.register_parameter(\"variational_cluster_logits\", torch.nn.Parameter(torch.randn(num_tasks, num_clusters)))\n", " \n", " # The Gaussian observational noise\n", " self.register_parameter(\"raw_noise\", torch.nn.Parameter(torch.tensor(0.0)))\n", " \n", " # Other info\n", " self.num_tasks = num_tasks\n", " self.num_clusters = num_clusters\n", " self.max_plate_nesting = 1\n", "\n", " def pyro_guide(self, function_dist, target):\n", " # Here we add the extra variational distribution for the cluster latent variable\n", " pyro.sample(\n", " self.name_prefix + \".cluster_logits\", # self.name_prefix is added by PyroQEP\n", " pyro.distributions.OneHotCategorical(logits=self.variational_cluster_logits).to_event(1)\n", " )\n", " return super().pyro_guide(function_dist, target)\n", "\n", " def pyro_model(self, function_dist, target):\n", " # Here we add the extra prior distribution for the cluster latent variable\n", " cluster_assignment_samples = pyro.sample(\n", " self.name_prefix + \".cluster_logits\", # self.name_prefix is added by PyroQEP\n", " pyro.distributions.OneHotCategorical(logits=self.prior_cluster_logits).to_event(1)\n", " )\n", " return super().pyro_model(function_dist, target, cluster_assignment_samples=cluster_assignment_samples)\n", "\n", " def forward(self, function_samples, cluster_assignment_samples=None):\n", " # For inference, cluster_assignment_samples will be passed in\n", " # This bit of code is for when we use the likelihood in the predictive mode\n", " if cluster_assignment_samples is None:\n", " cluster_assignment_samples = pyro.sample(\n", " self.name_prefix + \".cluster_logits\",\n", " pyro.distributions.OneHotCategorical(logits=self.variational_cluster_logits).to_event(1)\n", " )\n", " \n", " # Now we return the observational distribution, based on the function_samples and cluster_assignment_samples\n", " res = pyro.distributions.Normal(\n", " loc=(function_samples.unsqueeze(-2) * cluster_assignment_samples).sum(-1),\n", " scale=torch.nn.functional.softplus(self.raw_noise).sqrt()\n", " ).to_event(1)\n", " return res" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Constructing the PyroQEP model\n", "\n", "The PyroQEP model is essentially the same as the model we used in the simple example, except for two changes\n", "\n", "- We now will use our more complicated `ClusterGaussianLikelihood`\n", "- The latent function should be vector valued to correspond to the `k` latent functions. As a result, we will learn a batched variational distribution, and use a `UncorrelatedMultitaskVariationalStrategy` to convert the batched variational distribution into a `MultitaskMultivariateQExponential` distribution." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "POWER = 1.5\n", "class ClusterMultitaskQEPModel(qpytorch.models.pyro.PyroQEP):\n", " def __init__(self, train_x, train_y, num_functions=2, reparam=False):\n", " self.power = torch.tensor(POWER)\n", " num_data = train_y.size(-2)\n", "\n", " # Define all the variational stuff\n", " inducing_points = torch.linspace(0, 1, 64).unsqueeze(-1)\n", " variational_distribution = qpytorch.variational.CholeskyVariationalDistribution(\n", " num_inducing_points=inducing_points.size(-2),\n", " batch_shape=torch.Size([num_functions]),\n", " power=self.power\n", " )\n", " \n", " # Here we're using a UncorrelatedMultitaskVariationalStrategy - so that the output of the\n", " # QEP latent function is a MultitaskMultivariateQExponential\n", " variational_strategy = qpytorch.variational.UncorrelatedMultitaskVariationalStrategy(\n", " qpytorch.variational.VariationalStrategy(self, inducing_points, variational_distribution),\n", " num_tasks=num_functions,\n", " )\n", "\n", " # Standard initializtation\n", " likelihood = ClusterGaussianLikelihood(train_y.size(-1), num_functions)\n", " super().__init__(variational_strategy, likelihood, num_data=num_data)\n", " self.likelihood = likelihood\n", " self.num_functions = num_functions\n", "\n", " # Mean, covar\n", " self.mean_module = qpytorch.means.ZeroMean()\n", " self.covar_module = qpytorch.kernels.ScaleKernel(qpytorch.kernels.RBFKernel())\n", "\n", " def forward(self, x):\n", " mean_x = self.mean_module(x)\n", " covar_x = self.covar_module(x)\n", " res = qpytorch.distributions.MultivariateQExponential(mean_x, covar_x, power=self.power)\n", " return res" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This model can now be used to perform inference on cluster assignments, as well as make predictions using the inferred cluster assignments!" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "train_x = torch.linspace(0, 1, 100)\n", "train_y = torch.stack([\n", " torch.sin(train_x * (2 * math.pi)) + torch.randn(train_x.size()) * 0.2,\n", " torch.cos(train_x * (2 * math.pi)) + torch.randn(train_x.size()) * 0.2,\n", " torch.sin(train_x * (2 * math.pi)) + 2 * torch.cos(train_x * (2 * math.pi)) + torch.randn(train_x.size()) * 0.2,\n", " -torch.cos(train_x * (2 * math.pi)) + torch.randn(train_x.size()) * 0.2,\n", "], -1)\n", "num_tasks = train_y.size(-1)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "a0afade5ccb2409b85dad9dda5d33561", "version_major": 2, "version_minor": 0 }, "text/plain": [ " 0%| | 0/300 [00:00" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Set into eval mode and make predictions\n", "model.eval()\n", "with torch.no_grad():\n", " test_x = torch.linspace(0, 1, 51)\n", " predictions = model.likelihood(model(test_x)).rsample()\n", "print(predictions.shape) # num_samples x num_data x num_tasks\n", " \n", "fig, axs = plt.subplots(1, num_tasks, figsize=(4 * num_tasks, 3))\n", "for task, ax in enumerate(axs):\n", " ax.plot(train_x.detach().numpy(), train_y[:, task].detach().numpy(), 'k*')\n", " ax.plot(test_x.numpy(), predictions[:, :, task].T.numpy(), 'b', lw=1, alpha=0.2)\n", " ax.set_ylim([-3, 3])\n", " ax.legend(['Observed Data', 'Mean', 'Confidence'])\n", " ax.set_title(f'Task {task + 1}')\n", "\n", "fig.tight_layout()\n", "None" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAArMAAAE8CAYAAADT6TmLAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjcsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvTLEjVAAAAAlwSFlzAAAPYQAAD2EBqD+naQAALypJREFUeJzt3Qd01FX2wPGbhBQQCL2XIF16ETYgxBUWRMqGRUFEQUBUbBRRQUqIKKAuCFJEQMA9wgIbYEWjIIQAKiBdhUPoEFRC+bOEnrDJ/M99e2ZMIMFMmEzmN/P9nPPbzO83v/KYDZfrm/fu87PZbDYBAAAALMg/vxsAAAAA5BbJLAAAACyLZBYAAACWRTILAAAAyyKZBQAAgGWRzAIAAMCySGYBAABgWSSzAAAAsCySWQAAAFgWySy8QlhYmDz99NP53QwAsCziKKyKZBYe7ejRo/Lcc8/JvffeKyEhIVK0aFFp3bq1TJ8+Xa5fv+6WNly7dk3Gjx8vGzduFHf75JNPpG7duubPXrNmTZkxY4bb2wDA2nw5jn700Ufy2GOPSZUqVcTPz49ODy9VIL8bAGQnNjbWBKHg4GDp27ev1K9fX1JTU+W7776T1157Tfbv3y9z5851SxCOjo42rx988EG3/R/28ccfy/PPPy89evSQ4cOHy7fffiuvvPKKac8bb7zhtnYAsC5fj6PvvvuuXL58WVq0aCGnT59223PhXiSz8EjHjx+Xxx9/XKpWrSobNmyQ8uXLO9578cUX5ciRIyZIW9nVq1flnnvuyfI97S0ZPXq0dO7cWWJiYsyxQYMGSXp6ukyYMEGeffZZKV68uJtbDMBKfD2Oqk2bNjl6ZQsXLuzWtsF9GGYAj/Tee+/JlStXzNfsGQOwXY0aNWTIkCHZXq9fZ2nwutWiRYvM8RMnTjiO7dy5Uzp27CilSpWSggULSrVq1WTAgAHmPT2vdOnS5rX2Kui1uun97RISEuTRRx+VEiVKmK/wmjdvLqtXr87yuRpYX3jhBSlTpoxUqlQp2/bHx8fL//3f/5lzM9J/gDR4W/0fIAB5z9fjqNJEPqs/A7wLPbPwSF988YUZ39WqVas8fc7Zs2elQ4cOJtCOHDlSihUrZgLvypUrzft6XMdcDR48WLp37y5/+9vfzPGGDRuan/oVnY49q1ixorleewiWL18ukZGRsmLFCnNNRhqA9Z7jxo0zSWl29uzZY35qQM+oWbNm4u/vb95/8sknXf55APAevh5H4TtIZuFxLl26JL/++qv89a9/zfNnbdmyRf7zn//IN998kylxfPvtt81PDaraW6BBWAPvrQmk9mroV1g7duwwY9LsgfaBBx4w41pvDcLa6xAXFycBAQF3bJeO7dJztOcho6CgIClZsqT89ttvd/1nB+C9iKPwJQwzgEcGYVWkSJE8f5b2IKgvv/xSbt686dS1Fy5cMOPQevbsaSYYnD9/3mw6PEC/bjt8+LBJyjPSca9/lMjax8xq4poV/QrOXTOQAVgTcRS+hGQWHkfLxihNEPNaRESEqRag47h0rJf2Bi9cuFBSUlL+8FqdPGGz2WTs2LHmK6+MW1RUlOPrt4x0HFlO6JgznXGclRs3bpj3ASA7xFH4EoYZwCODcIUKFWTfvn25vkd2A/7T0tJuO0+rBWzbts2ML1u7dq2ZtDBlyhRz7E6zX7WygBoxYoTpic2KTrDIKKdJqE7W0LZqMpxxqIEmuNrzq58PAGSHOApfQjILj9SlSxdT+3Dr1q0SHh7u9PX2slUXL150DCVQJ0+ezPL8P/3pT2Z75513ZMmSJdKnTx9ZunSpPPPMM9kmxjqxQgUGBkr79u3FlRo3buyYIfzII484juu+JtH29wEgO74eR+E7GGYAj/T666+byVcaBM+cOZPlija6ek12qlevbn5u3rzZcUxnvX766aeZztPJXzpUICN7omgfalCoUCFHQM9Ie0y1+LcubpBVMe5z585Jbj300ENmspjOAM5I97U9Wn8WAO7E1+MofAc9s/BIGkT1v+x79epllnPNuHKNViD417/+dcdlCbVMjFYZGDhwoFnlRiddLViwwIxnTUxMdJynQXn27Nmm6oA+U8fpzps3z3xFZ+8R1aEB9913nyxbtkxq1aplkkxti26zZs0ylQsaNGhgJndpL4P+o6E9Ib/88ov8+OOPufrz6zN1cQStK6ur9+gwBl0B7LPPPjO9HtoGACCO3pkOH7PHYZ3k+9NPPzmq1XTr1s1RHgwWZwM82KFDh2yDBg2yhYWF2YKCgmxFihSxtW7d2jZjxgzbjRs3HOdVrVrV1q9fv0zX7tq1y9ayZUtzXZUqVWxTp061LVy4ULsPbMePHzfn7N6929a7d2/zfnBwsK1MmTK2Ll262Hbu3JnpXlu2bLE1a9bM3Euvj4qKcrx39OhRW9++fW3lypWzBQYG2ipWrGjuERMT4zjH/twdO3Y49eefO3eurXbt2ua51atXt33wwQe29PR0pz9HAL7Ll+Oo/nn0mqw2vR+8g5/+T34n1AAAAEBuMGYWAAAAlkUyCwAAAMsimQUAAIBl5Wsyq+U+unbtagrAaw26f//73394zcaNG6Vp06YSHBxsCtIvWrTILW0FAE9EHAXg6/I1mdV6dY0aNTLljXLi+PHjpr7mn//8Z9m7d68MHTrU1M/TVZsAwBcRRwH4Oo+pZqA9s6tWrZLIyMhsz3njjTckNjY20zKnjz/+uCnCvGbNGje1FAA8E3EUgC+y1KIJWoj+1uXutJi89tBmR1cfsa9AonQp0AsXLkjJkiWzXV4PAO6G9hHoAhw6hMrf37OmJhBHAXhbHLVUMpuUlCRly5bNdEz3L126JNevXzerJt1q0qRJEh0d7cZWAsD/nDp1SipVquRRHwdxFIC3xVFLJbO5MWrUKBk+fLhjPzk52Sxzqh+OLlkKAK6m/4FduXJlKVKkiFd8uMRRAJ4cRy2VzJYrV86se5+R7mtSmlWvrNKqB7rdSq8hmQWQlzxxKBNxFIC3xVHPGsz1B8LDwyUuLi7TsXXr1pnjAADiKADfk6/J7JUrV0yJLd3spbf0dWJiouOrrb59+zrOf/755+XYsWPy+uuvS0JCgsyePVuWL18uw4YNy7c/AwDkJ+IoAF+Xr8nszp07pUmTJmZTOrZVX48bN87snz592pHYqmrVqpnSXNobq/Vpp0yZIvPnzzcVDQDAFxFHAfg6j6kz684BxaGhoWYiGGNmARBniKMArJ2vWWrMLAAAAJARySwAAAAsi2QWAAAAlkUyCwAAAMuy1KIJAAAA3i5sZKx4mxOTO+fZvemZBQAAgGWRzAIAAMCyGGYAAHALb/zqNK+/PgXwx+iZBQAAgGWRzAIAAMCySGYBAABgWSSzAAAAsCySWQAAAFgWySwAAAAsi2QWAAAAlkUyCwAAAMsimQUAAIBlkcwCAADAskhmAQAAYFkkswAAALAsklkAAABYFsksAAAALItkFgAAAJZFMgsAAADLIpkFAACAZZHMAgAAwLJIZgEAAGBZLktmd+7cKZs3b3bV7QAAAIA/VEBc5KmnnpJDhw5JWlqaq24JAAAAuCeZjYuLk5s3b7rqdgAAAID7ktkKFSq46lawgLCRseJtTkzunN9NAAAA7khmdSjBqlWr5MCBA2a/bt26EhkZKQUKuCw3BgAAAP6Q09nn/v37pVu3bpKUlCS1a9c2x959910pXbq0fPHFF1K/fn1nbwkAAAC4p5rBM888I/Xq1ZNffvlFdu/ebbZTp05Jw4YN5dlnn81dKwAAAAB39Mzu3bvXlOEqXry445i+fuedd+T+++/PTRsAAAAA9/TM1qpVS86cOXPb8bNnz0qNGjWcbsCsWbMkLCxMQkJCpGXLlrJ9+/Y7nj9t2jQzvKFgwYJSuXJlGTZsmNy4ccPp5wKANyGWAvBVOUpmL1265NgmTZokr7zyisTExJihBrrp66FDh5qxs85YtmyZDB8+XKKiosxwhUaNGknHjh1NYpyVJUuWyMiRI835Ovnsk08+Mfd48803nXouAHgTYikAX5ajYQbFihUTPz8/x77NZpOePXs6jum+6tq1q1OLJkydOlUGDRok/fv3N/tz5syR2NhYWbBggUlab7VlyxZp3bq1PPHEE2Zfe3R79+4tP/zwQ46fCQDehlgKwJflKJmNj493+YNTU1Nl165dMmrUKMcxf39/ad++vWzdujXLa1q1aiWfffaZGYrQokULOXbsmHz11Vdm9bHspKSkmM1Oe5cBwFu4I5YSRwFYPpmNiIhw+YPPnz9venHLli2b6bjuJyQkZHmN9sjqdQ888IDpDf7vf/8rzz///B2HGeiwiOjoaJe3HwA8gTtiKXEUgFdNAFMXL16UKVOmmDJdun3wwQeSnJwseW3jxo0yceJEmT17thlju3LlSjMsYcKECdleo70V2jb7pmXEAMCXORtLiaMAvKo0l5bl0klaWk1Av56yj9fS0lzffPONNG3aNEf3KVWqlAQEBNxWGUH3y5Url+U1Y8eONV+DaQKtGjRoIFevXjX1bUePHm2+WrtVcHCw2YC84o1L+yqW97UGd8RS4igAr+qZ1VJYugLYiRMnzH/N63b8+HHp0qWLqWiQU0FBQdKsWTOJi4tzHEtPTzf74eHhWV5z7dq124KsBvGMk9AAwJcQSwH4ulz1zM6bN08KFPj9Un39+uuvS/PmzZ26l5bl6tevn7lOe3m1hqz2DtirG/Tt21cqVqxoxmvZqyVoL3CTJk1MTdojR46YHgY9bk9qAcDXEEsB+DKnk9miRYtKYmKi1KlTJ9NxHYtapEgRp+7Vq1cvOXfunIwbN06SkpKkcePGsmbNGsdEBn1Oxp7YMWPGmHJg+vPXX3+V0qVLm0RWhzgAgK8ilgLwZX42J7+f1wUTVq1aJX//+99NeRf1/fffy2uvvSY9evQwvaueTEtzhYaGmslgmpgjd7xxnGhux4h642ehGDObe94eZ3L75+PvCuC7f1dOOPlvrDNxxumeWU1itXdUhwBoORcVGBgogwcPlsmTJzt7OwAAACDXnEpmtZbhtm3bZPz48WYc69GjR83x6tWrS6FChXLfCgAAACCvk1mdZNWhQwc5cOCAVKtWzZRzAQAAACxTmqt+/fpm6UMAAADAcsns22+/LSNGjJAvv/xSTp8+bQboZtwAAAAAd3F6AtgjjzxifurCCToRzE6LIui+jqsFAAAAPDKZjY+Pz5uWAAAAAHmZzGrva4UKFSQ1NVVq166daRUwAAAAwGPHzB4/flwaNmxoVv7Sn1qOS5e2BQAAADw+mdUVvnSRhM8++0xiYmKkUqVK8txzz+Vt6wAAAIA7yPE4ge+++84ksQ888IDZ/9Of/mQS2qtXr8o999yT09sAAAAA7u+ZPXv2rNSsWdOxX758eSlYsKA5DgAAAHh0z6yW3bpy5YpJYO38/f3l8uXLmerLFi1a1PWtBAAAAO4mmdVKBrVq1brtWJMmTRyvqTMLAAAAj0xmqS8LAAAAyyazERERedsSAAAAIK8mgAEAAACehmQWAAAAlkUyCwAAAMsimQUAAIBvJLM3b96UAgUKyL59+/KuRQAAAEBeJLOBgYFSpUoVSUtLc+YyAAAAwDOGGYwePVrefPNNuXDhQt60CAAAAHB1nVm7mTNnypEjR6RChQpStWpVueeeezK9v3v3bmdvCQAAALgnmY2MjMzdkwAAAID8TmajoqJc3QYAAADAfaW5Ll68KPPnz5dRo0Y5xs7q8IJff/01d60AAAAA3NEz+9NPP0n79u0lNDRUTpw4IYMGDZISJUrIypUrJTExUf7xj3/kph0AAABA3vfMDh8+XJ5++mk5fPiwhISEOI4/8sgjsnnzZudbAAAAALgrmd2xY4c899xztx2vWLGiJCUl5bYdAAAAQN4ns8HBwXLp0qXbjh86dEhKly7tfAsAAAAAdyWz3bp1k7feesssbav8/PzMWNk33nhDevTokdt2AAAAAHmfzE6ZMkWuXLkiZcqUkevXr0tERITUqFFDihQpIu+8847zLQAAAADcVc1AqxisW7dOvv/+e/nxxx9NYtu0aVNT4QAAAADw6GRWS2/16tVLWrdubTa71NRUWbp0qfTt29fVbQQAAABcM8ygf//+kpycfNvxy5cvm/ecNWvWLAkLCzNlvlq2bCnbt2//wwUbXnzxRSlfvryZjFarVi356quvnH4uAHgTYikAX+V0z6zNZjOTvm71yy+/mCEIzli2bJmpWztnzhyTyE6bNk06duwoBw8eNGNyb6W9v3/5y1/MezExMaYc2MmTJ6VYsWLO/jEAwGsQSwH4shwns02aNDFJrG7t2rWTAgV+vzQtLU2OHz8uDz/8sFMPnzp1qllBzN6jq0ltbGysLFiwQEaOHHnb+Xpcl8/dsmWLBAYGmmPaqwsAvoxYCsCX5TiZjYyMND/37t1rek8LFy7seC8oKMgklc6U5tJe1l27dsmoUaMcx/z9/c1Esq1bt2Z5zerVqyU8PNwMM/j8889NXdsnnnjClAULCAjI8pqUlBSz2WVVIxcArModsZQ4CsArktmoqCjzU5PWxx9/3IxXvRvnz583Pbply5bNdFz3ExISsrzm2LFjsmHDBunTp48ZJ3vkyBF54YUXTM1be/tuNWnSJImOjr6rtgKAp3JHLCWOAvCqCWAPPfSQnDt3zrGvE7aGDh0qc+fOlbyWnp5uxsvqs5o1a2aqKowePdoMT8iO9lbohDX7durUqTxvJwB4MmdjKXEUgFdNANOvop599ll56qmnJCkpyXyVVb9+fVm8eLHZHzduXI7uU6pUKfN11pkzZzId1/1y5cpleY1WMNCxshm/Bqtbt655rn7VpsMdbqU9yHfbiwwAnsodsZQ4CsCremb37dsnLVq0MK+XL18uDRo0MBOyNJldtGhRju+jwVJ7BOLi4jL1Fui+juXKita11a/D9Dy7Q4cOmcCcVSILAN6OWArA1zmdzOqYKntP5/r166Vbt27mdZ06deT06dNO3UvLcs2bN08+/fRTOXDggAwePFiuXr3qqG6gCzBknNSg72s1gyFDhpgkVisfTJw40UxiAABfRSwF4MucHmZQr149M66qc+fOZlnbCRMmmOO//fablCxZ0ql76TgtHX+rQxP0663GjRvLmjVrHBMZEhMTzaxcu8qVK8vatWtl2LBh0rBhQ1NnVhNbnYELAL6KWArAlzmdzL777rvSvXt3ef/996Vfv37SqFEjR6kX+/ADZ7z00ktmy8rGjRtvO6ZDELZt2+b0cwDAmxFLAfgqp5PZBx980JSC0XqtxYsXdxzXSWGFChVydfsAAAAA1yWzSmfAZkxkFStxAQAAwOOT2WrVqpklbbOjxbgBAAAAj0xmdYGEW6sb7Nmzx0zceu2111zZNgAAAMC1yaxWD8jKrFmzZOfOnc7eDgAAAHBfndnsdOrUSVasWOGq2wEAAADuS2ZjYmKkRIkSrrodAAAA4PphBk2aNMk0Acxms5kFD3Txg9mzZzt7OwAAAMB9yWxkZGSmfV2hq3Tp0qb+rC5pCwAAAHhsMhsVFZU3LQEAAADyIpnV1b5yqmjRos62AQAAAMi7ZLZYsWJ3XCjBPnZWz0lLS8tdSwAAAIC8SGbj4+OdvS8AAADgGclsRERE3rcEAAAAyKs6s4cPH5bevXtnOX42OTlZnnjiCTl27JizzwcAAADyPpl9//33pXLlyllO8AoNDTXv6TkAAACAxyWzmzZtksceeyzb93v27CkbNmxwVbsAAAAA1yWziYmJUqZMmWzfL1WqlJw6dSqntwMAAADcl8zqUIKjR49m+/6RI0eoMQsAAADPTGbbtm0rM2bMyPb9Dz/8UNq0aeOqdgEAAACuS2ZHjRolX3/9tTz66KOyfft2U8FAtx9++EF69Ogha9euNecAAAAAHlVnVjVp0kRiYmJkwIABsmrVqkzvlSxZUpYvXy5NmzbNizYCAAAAd5fMqi5dusjJkydlzZo1ZoysLmFbq1Yt6dChgxQqVMiZWwEAAADuTWZVwYIFpXv37nz0AAAAsM6YWQAAAMDTkMwCAADAskhmAQAAYFkkswAAAPDuCWCXLl3K8Q2LFi16N+0BAAAAXJvMFitWTPz8/O54jpbp0nPS0tJy/nQAAAAgr5PZ+Pj4u3kGAAAAkH/JbERERN48HQAAAHDnogl2165dk8TERElNTc10vGHDhnfTHgAAACDvktlz585J//795euvv87yfcbMAgAAwGNLcw0dOlQuXrwoP/zwg1nads2aNfLpp59KzZo1ZfXq1XnTSgAAAMAVyeyGDRtk6tSp0rx5c/H395eqVavKk08+Ke+9955MmjRJcmPWrFkSFhYmISEh0rJlS9m+fXuOrlu6dKmpoBAZGZmr5wKAtyCOAvBVTiezV69elTJlypjXxYsXN8MOVIMGDWT37t1ON2DZsmUyfPhwiYqKMtc3atRIOnbsKGfPnr3jdSdOnJARI0ZImzZtnH4mAHgT4igAX+Z0Mlu7dm05ePCgea2J58cffyy//vqrzJkzR8qXL+90A7SXd9CgQWYc7n333WfuU6hQIVmwYEG21+i43D59+kh0dLTce++9Tj8TALwJcRSAL3M6mR0yZIicPn3avNbeVJ0IVqVKFfnwww9l4sSJTt1LKyHs2rVL2rdv/3uD/P3N/tatW7O97q233jK9wwMHDvzDZ6SkpJgVzDJuAOAtiKMAfJ3T1Qx0fKxds2bN5OTJk5KQkGAS2lKlSjl1r/Pnz5te1rJly2Y6rvt6z6x899138sknn8jevXtz9Awdx6s9uADgjYijAHyd0z2zt9IhAU2bNnU6kc2Ny5cvy1NPPSXz5s3L8fNGjRolycnJju3UqVN53k4A8FTEUQDi6z2z2pO6aNEiiYuLM5O00tPTb6t2kFOakAYEBMiZM2cyHdf9cuXK3Xb+0aNHzcSvrl27Oo7Zn1+gQAEzlrd69eqZrgkODjYbAHgj4igAX1cgN2NmNZnt3Lmz1K9f35TGyq2goCAzVEETY3t5LU1Odf+ll1667fw6derIzz//nOnYmDFjTE/D9OnTpXLlyrluCwBYEXEUgK9zOpnV2q7Lly+XRx55xCUN0LJc/fr1M3VrW7RoIdOmTTPlv7S6gerbt69UrFjRjH3VOrSaQGdUrFgx8/PW4wDgK4ijAHxZgdz0AtSoUcNlDejVq5epVTtu3DhJSkqSxo0bm1XF7JPCEhMTTYUDAABxFADuOpl99dVXzVf6M2fOvKshBhnpkIKshhWojRs33vFaHfIAAL6OOArAVzmdzGpprPj4eFNftl69ehIYGJjp/ZUrV7qyfQAAAIDrklkdo9q9e3dnLwMAAADyP5lduHCh61sBAAAAuCOZtdNJW1rXVdWuXVtKly6d21sBAAAAueJ0mQAtmzVgwAApX768tG3b1mwVKlSQgQMHyrVr13LXCgAAAMAdyazWM9y0aZN88cUXcvHiRbN9/vnn5phWOgAAAAA8dpjBihUrJCYmRh588EHHMV1AoWDBgtKzZ0/56KOPXN1GAAAAwDU9szqUwL6gQUZlypRhmAEAAAA8O5kNDw+XqKgouXHjhuPY9evXJTo62rwHAAAAeOwwA139q2PHjlKpUiVp1KiROfbjjz9KSEiIrF27Ni/aCAAAALgmma1fv74cPnxYFi9eLAkJCeZY7969pU+fPmbcLAAAAODRdWYLFSokgwYNcn1rAAAAAFcns6tXr5ZOnTpJYGCgeX0n3bp1c+b5AAAAQN4ms5GRkZKUlGQqFujr7Pj5+UlaWlruWwMAAAC4OplNT0/P8jUAAABgqdJc//jHPyQlJeW246mpqeY9AAAAwGOT2f79+0tycvJtxy9fvmzeAwAAADw2mbXZbGZs7K1++eUXCQ0NdVW7AAAAANeV5mrSpIlJYnVr166dFCjw+6U66ev48ePy8MMP5/R2AAAAgPuSWXsVg71795oVwAoXLux4LygoSMLCwqRHjx533yIAAADA1clsVFSU+alJa69evczytQAAAIClVgDr169f3rQEAAAAyOtkVsfHfvDBB7J8+XJJTEw0JbkyunDhgrO3BAAAANxTzSA6OlqmTp1qhhpoia7hw4fL3/72N/H395fx48fnrhUAAACAO5LZxYsXy7x58+TVV181FQ169+4t8+fPl3Hjxsm2bdty0wYAAADAPclsUlKSNGjQwLzWigb2BRS6dOkisbGxuWsFAAAA4I5ktlKlSnL69Gnzunr16vLNN9+Y1zt27JDg4ODctAEAAABwTzLbvXt3iYuLM69ffvllGTt2rNSsWVP69u0rAwYMyF0rAAAAAHdUM5g8ebLjtU4Cq1KlimzdutUktF27ds1NGwAAAAD3JLO3Cg8PNxsAAADgkcns6tWrpVOnThIYGGhe30m3bt1c1TYAAADg7pPZyMhIU8WgTJky5nV2/Pz8zKIKAAAAgMcks+np6Vm+BgAAACxTzeDmzZvSrl07OXz4cN61CAAAAMiLZFbHzP7000/OXAIAAAB4Tp3ZJ598Uj755JO8aQ0AAACQl8nsf//7X/noo4+kefPm8txzz8nw4cMzbbkxa9YsCQsLk5CQEGnZsqVs374923PnzZsnbdq0keLFi5utffv2dzwfAHwBcRSAr3K6zuy+ffukadOm5vWhQ4duq2bgrGXLlpkkeM6cOSaRnTZtmnTs2FEOHjxoqifcauPGjdK7d29p1aqVSX7fffdd6dChg+zfv18qVqzo9PMBwOqIo7C6sJGx4o1OTO6c303wCU4ns/Hx8S5twNSpU2XQoEHSv39/s69JbWxsrCxYsEBGjhx52/mLFy/OtD9//nxZsWKFWWJXl9QFAF9DHAXgy5weZuBKqampsmvXLjNUwNEgf3+zr0vk5sS1a9dMlYUSJUpk+X5KSopcunQp0wYA3oI4CsDX5Wo52507d8ry5cslMTHRBNKMVq5cmeP7nD9/3iyyULZs2UzHdT8hISFH93jjjTekQoUKmRLijCZNmiTR0dE5bhMAWAlxFICvc7pndunSpWa86oEDB2TVqlWmV1THq27YsEFCQ0PFnSZPnmzao+3Q8bNZGTVqlCQnJzu2U6dOubWNAODJiKMAfK5nduLEifLBBx/Iiy++KEWKFJHp06dLtWrVTGWD8uXLO3WvUqVKSUBAgJw5cybTcd0vV67cHa/9+9//boLw+vXrpWHDhtmeFxwcbDYA8EbEUQC+zume2aNHj0rnzv+bnRcUFCRXr141VQyGDRsmc+fOdepeen2zZs3M5K2My+Xqfnh4eLbXvffeezJhwgRZs2aNKREGAL6KOArA1zmdzGpt18uXL5vXWgpLS3WpixcvmslYztKyXFo79tNPPzVDFwYPHmwSZHt1A61QoEMF7LQU19ixY021A61Nm5SUZLYrV644/WwA8AbEUQC+zOlhBm3btpV169ZJgwYN5LHHHpMhQ4aY8bJ6rF27dk43oFevXnLu3DkZN26cSUobN25selztk8J0kplWOLDTBRt00tmjjz6a6T5RUVEyfvx4p58PAFZHHAXgy3KczGoPbP369WXmzJly48YNc2z06NESGBgoW7ZskR49esiYMWNy1YiXXnrJbFnRRRIyOnHiRK6eAQDejDgKwFflOJnVSVb333+/PPPMM/L444+bY9pjmtXCBgAAAIBHJbObNm2ShQsXyquvvmome2lPrCa2bdq0ydsWAgDghVjCFXDzBDBNWnXS1enTp2XGjBnm6/6IiAipVauWmZSl410BAAAAj65mcM8995hKA9pTe+jQITMJbNasWVKlShXp1q1b3rQSAAAAcEUym1GNGjXkzTffNBO/dAGF2NjYu7kdAAAAkLeluew2b95shh2sWLHCTATr2bOnDBw4MLe3AwAAAPI2mf3tt99k0aJFZjty5Ii0atVKPvzwQ5PI6vADAAAAwCOT2U6dOsn69evNOuC6KteAAQOkdu3aeds6AAAAwBXJrC6OEBMTI126dJGAgICcXgYAAADkfzK7evXqvGsFAAAA4O5qBgAAAEB+IpkFAACAZZHMAgAAwLJIZgEAAGBZJLMAAACwLJJZAAAAWBbJLAAAACyLZBYAAACWRTILAAAAyyKZBQAAgGWRzAIAAMCySGYBAABgWSSzAAAAsCySWQAAAFgWySwAAAAsq0B+N8AqwkbGijc6MblzfjcBAAAg1+iZBQAAgGXRMwvAZfgGAwDgbvTMAgAAwLJIZgEAAGBZJLMAAACwLJJZAAAAWBbJLAAAACyLZBYAAACWRTILAAAAy/KIZHbWrFkSFhYmISEh0rJlS9m+ffsdz//Xv/4lderUMec3aNBAvvrqK7e1FQA8EXEUgK/K92R22bJlMnz4cImKipLdu3dLo0aNpGPHjnL27Nksz9+yZYv07t1bBg4cKHv27JHIyEiz7du3z+1tBwBPQBwF4MvyPZmdOnWqDBo0SPr37y/33XefzJkzRwoVKiQLFizI8vzp06fLww8/LK+99prUrVtXJkyYIE2bNpWZM2e6ve0A4AmIowB8Wb4uZ5uamiq7du2SUaNGOY75+/tL+/btZevWrVleo8e1Jzcj7cn997//neX5KSkpZrNLTk42Py9duuRUW9NTrok3cvZz8ObPg8/i7j8Pb/y9yM1nYT/fZrNJXiOO5j9ix91/HsQO7/88LuVhHM3XZPb8+fOSlpYmZcuWzXRc9xMSErK8JikpKcvz9XhWJk2aJNHR0bcdr1y58l213VuETsvvFngOPgs+D1f/bly+fFlCQ0MlLxFH8x+xg8+D3438jaP5msy6g/b6ZuzJTU9PlwsXLkjJkiXFz89PPI3+l4gm2qdOnZKiRYuKL+Oz4POw6u+G9iRoAK5QoYJ4A+KotXn63xd34rPwzjiar8lsqVKlJCAgQM6cOZPpuO6XK1cuy2v0uDPnBwcHmy2jYsWKiafTXyxP/OXKD3wWfB5W/N3I6x5ZO+KoNX8/8gOfB5+Ft8bRfJ0AFhQUJM2aNZO4uLhMPae6Hx4enuU1ejzj+WrdunXZng8A3ow4CsDX5fswAx0C0K9fP2nevLm0aNFCpk2bJlevXjXVDVTfvn2lYsWKZuyrGjJkiERERMiUKVOkc+fOsnTpUtm5c6fMnTs3n/8kAJA/iKMAfFm+J7O9evWSc+fOybhx48wkrsaNG8uaNWsck7wSExNNhQO7Vq1ayZIlS2TMmDHy5ptvSs2aNU0lg/r164s30CERWnP31qERvojPgs+D342cIY7yd4VYyr8rvvxvrJ/NHbVjAAAAAG9cNAEAAADILZJZAAAAWBbJLAAAACyLZBYAAACWRTLrYWbNmiVhYWESEhIiLVu2lO3bt4sv2rx5s3Tt2tWs/KErtWnFCl+lZenuv/9+KVKkiJQpU0YiIyPl4MGD4os++ugjadiwoaPIt9aX/vrrr/O7WfAwxNH/IY7+jjjq3XGUZNaDLFu2zNSL1FIZu3fvlkaNGknHjh3l7Nmz4mu01rD++fUfJV+3adMmefHFF2Xbtm1mgZCbN29Khw4dzGfkaypVqiSTJ0+WXbt2mfrSDz30kPz1r3+V/fv353fT4CGIo78jjv6OOOrdcZTSXB5Ee2K1B27mzJmO1dB03eSXX35ZRo4cKb5Ke2ZXrVpleiQhpi6z9tBqcG7btq3PfyQlSpSQ999/XwYOHOjznwWIo8TRnCGOelccpWfWQ6Smppr/Smrfvr3jmC4Woftbt27N17bBsyQnJzuCjy9LS0szKwBq7xPLWUMRR5FTxFHviqP5vgIY/uf8+fPml8q+8pmd7ickJPAxwdFbP3ToUGndurXXrHrnrJ9//tkE3Rs3bkjhwoVNr/19992X382CByCOIieIo+J1cZRkFrAQHTu7b98++e6778RX1a5dW/bu3Wt6VmJiYqRfv35myIWVAzEA9yGOitfFUZJZD1GqVCkJCAiQM2fOZDqu++XKlcu3dsFzvPTSS/Lll1+aGco6gN9XBQUFSY0aNczrZs2ayY4dO2T69Ony8ccf53fTkM+Io/gjxFHvjKOMmfWgXyz9hYqLi8v0VYjuW3kcC+6ezWYzAVi/BtqwYYNUq1aNjzUD/XuSkpLCZwLiKIijPhpH6Zn1IFqWS7v6mzdvLi1atJBp06aZQdn9+/cXX3PlyhU5cuSIY//48ePmKxGd9FSlShXxta/ElixZIp9//rmpNZuUlGSOh4aGSsGCBcWXjBo1Sjp16mR+By5fvmw+l40bN8ratWvzu2nwEMTR3xFHf0cc9fI4aoNHmTFjhq1KlSq2oKAgW4sWLWzbtm2z+aL4+Hib/nreuvXr18/ma7L6HHRbuHChzdcMGDDAVrVqVfP3o3Tp0rZ27drZvvnmm/xuFjwMcfR/iKO/I456dxylziwAAAAsizGzAAAAsCySWQAAAFgWySwAAAAsi2QWAAAAlkUyCwAAAMsimQUAAIBlkcwCAADAskhmAQAAYFkks/A5Dz74oAwdOtTtzx0/frw0btzY7c8FAFcjjsKTkMzCI/n5+d1x08TQXU6cOGGeuXfvXrc9EwDuFnEUvqJAfjcAyMrp06cdr5ctWybjxo2TgwcPOo4VLlyYDw4A7oA4Cl9Bzyw8Urly5RxbaGio6WGw71+9elX69OkjZcuWNUnt/fffL+vXr890/ezZs6VmzZoSEhJiznv00UezfVZsbKx5xuLFi3PUto0bN5r2xMXFSfPmzaVQoULSqlWrTMm2mjx5snl2kSJFZODAgXLjxo3b7jV//nypW7euaWedOnVMu+0GDBggDRs2lJSUFLOfmpoqTZo0kb59++aonQB8G3GUOOozbICHW7hwoS00NNSxv3fvXtucOXNsP//8s+3QoUO2MWPG2EJCQmwnT5407+/YscMWEBBgW7Jkie3EiRO23bt326ZPn+64PiIiwjZkyBDzevHixbYiRYrYvvjii2yff/z4cZv+VdmzZ4/Zj4+PN/stW7a0bdy40bZ//35bmzZtbK1atXJcs2zZMltwcLBt/vz5toSEBNvo0aPNcxo1auQ457PPPrOVL1/etmLFCtuxY8fMzxIlStgWLVpk3r98+bLt3nvvtQ0dOtTsjxgxwhYWFmZLTk524acLwBcQR4mj3oxkFpYLwlmpV6+ebcaMGea1JoVFixa1Xbp0Kctz7cnszJkzzX01Ib2T7JLZ9evXO86JjY01x65fv272w8PDbS+88EKm+2jymzGZrV69ukm4M5owYYK51m7Lli22wMBA29ixY20FChSwffvtt3dsKwBkhThKHPVmDDOA5Vy5ckVGjBhhvp4vVqyYGWpw4MABSUxMNO//5S9/kapVq8q9994rTz31lBk+cO3atUz3iImJkWHDhsm6deskIiIiV+3QIQB25cuXNz/Pnj1rfmp7WrZsmen88PBwx2sdKnH06FEz/EDbb9/efvttczzjNfpnnTBhgrz66qvywAMP5KqtAJARcRTehGQWlqPJ3apVq2TixIny7bffmioDDRo0MGNKlY5R3b17t/zzn/80SaZOHmvUqJFcvHjRcQ8de1q6dGlZsGCBfjuRq3YEBgY6XusYWpWenp7jf0jUvHnzTPvt2759+2Tbtm2O8/R+33//vQQEBMiRI0dy1U4AuBVxFN6EZBaWo8nd008/Ld27dzdJrE5y0PJZGRUoUEDat28v7733nvz000/m/Q0bNjjer169usTHx8vnn38uL7/8ssvbqL3GP/zwQ6ZjGZNUnRhWoUIFOXbsmNSoUSPTVq1aNcd577//viQkJMimTZtkzZo1snDhQpe3FYDvIY7Cm1CaC5ajVQpWrlwpXbt2NT2iY8eOzdQj+uWXX5oksW3btlK8eHH56quvzPu1a9fOdJ9atWqZhFaLf2vyO23aNJe1cciQISbh1moHrVu3NkMd9u/fb4Y+2EVHR8srr7xiKik8/PDDpmrBzp075T//+Y8MHz5c9uzZY3qVdUiE3mPq1KnmvjosIuN9AMBZxFHiqDehZxaWo0mdJqlaDksT2o4dO0rTpk0d7+s4Wk12H3roIdNDOmfOHDPkoF69erfdSxNc7bHV93VMqqv06tXLJNmvv/66NGvWTE6ePCmDBw/OdM4zzzxjSnNpb6v2MGuSumjRItMzq2W8nnzySZMQ659RPfvss/LnP//ZjANOS0tzWVsB+B7iKHHUm/jpLLD8bgQAAACQG/TMAgAAwLJIZgEAAGBZJLMAAACwLJJZAAAAWBbJLAAAACyLZBYAAACWRTILAAAAyyKZBQAAgGWRzAIAAMCySGYBAABgWSSzAAAAEKv6fwTv7E6VofmZAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Infer cluster assignments\n", "variational_clusters = pyro.distributions.Categorical(logits=model.likelihood.variational_cluster_logits).probs\n", "\n", "# Plot cluster assignments\n", "fig, axs = plt.subplots(1, model.num_functions, figsize=(4 * model.num_functions, 3))\n", "for function, ax in enumerate(axs):\n", " index = torch.arange(num_tasks)\n", " ax.bar(index, variational_clusters[:, function].detach().numpy())\n", " ax.set(ylim=[0, 1], title=f\"Cluster {function}\", xlabel=\"Task Index\")\n", "axs[0].set_ylabel(\"Variational Cluster Prob.\")\n", "None" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.18" } }, "nbformat": 4, "nbformat_minor": 4 }