{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Sparse Q-Exponential Process Regression (SQEPR)\n", "\n", "## Overview\n", "\n", "In this notebook, we'll overview how to use [SGPR](http://proceedings.mlr.press/v5/titsias09a/titsias09a.pdf) in which the inducing point locations are learned." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import math\n", "import torch\n", "import qpytorch\n", "import tqdm.notebook as tqdm\n", "from matplotlib import pyplot as plt\n", "\n", "# Make plots inline\n", "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For this example notebook, we'll be using the `elevators` UCI dataset used in the paper. Running the next cell downloads a copy of the dataset that has already been scaled and normalized appropriately. For this notebook, we'll simply be splitting the data using the first 80% of the data as training and the last 20% as testing.\n", "\n", "**Note**: Running the next cell will attempt to download a ~400 KB dataset file to the current directory." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import urllib.request\n", "import os\n", "from scipy.io import loadmat\n", "from math import floor\n", "\n", "\n", "# this is for running the notebook in our testing framework\n", "smoke_test = ('CI' in os.environ)\n", "\n", "\n", "if not smoke_test and not os.path.isfile('../elevators.mat'):\n", " print('Downloading \\'elevators\\' UCI dataset...')\n", " urllib.request.urlretrieve('https://drive.google.com/uc?export=download&id=1jhWL3YUHvXIaftia4qeAyDwVxo6j1alk', '../elevators.mat')\n", "\n", "\n", "if smoke_test: # this is for running the notebook in our testing framework\n", " X, y = torch.randn(1000, 3), torch.randn(1000)\n", "else:\n", " data = torch.Tensor(loadmat('../elevators.mat')['data'])\n", " X = data[:, :-1]\n", " X = X - X.min(0)[0]\n", " X = 2 * (X / X.max(0)[0]) - 1\n", " y = data[:, -1]\n", "\n", "\n", "train_n = int(floor(0.8 * len(X)))\n", "train_x = X[:train_n, :].contiguous()\n", "train_y = y[:train_n].contiguous()\n", "\n", "test_x = X[train_n:, :].contiguous()\n", "test_y = y[train_n:].contiguous()\n", "\n", "if torch.cuda.is_available():\n", " train_x, train_y, test_x, test_y = train_x.cuda(), train_y.cuda(), test_x.cuda(), test_y.cuda()" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "torch.Size([16599, 18])" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X.size()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Defining the SQEPR Model\n", "\n", "We now define the QEP model. For more details on the use of QEP models, see our simpler examples. This model constructs a base scaled RBF kernel, and then simply wraps it in an `InducingPointKernel`. Other than this, everything should look the same as in the simple QEP models." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "from qpytorch.means import ConstantMean\n", "from qpytorch.kernels import ScaleKernel, RBFKernel, InducingPointKernel\n", "from qpytorch.distributions import MultivariateQExponential\n", "\n", "POWER = 1.0\n", "class QEPRegressionModel(qpytorch.models.ExactQEP):\n", " def __init__(self, train_x, train_y, likelihood):\n", " super(QEPRegressionModel, self).__init__(train_x, train_y, likelihood)\n", " self.power = torch.tensor(POWER)\n", " self.mean_module = ConstantMean()\n", " self.base_covar_module = ScaleKernel(RBFKernel())\n", " self.covar_module = InducingPointKernel(self.base_covar_module, inducing_points=train_x[:500, :].clone(), likelihood=likelihood)\n", "\n", " def forward(self, x):\n", " mean_x = self.mean_module(x)\n", " covar_x = self.covar_module(x)\n", " return MultivariateQExponential(mean_x, covar_x, power=self.power)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "likelihood = qpytorch.likelihoods.QExponentialLikelihood(power=torch.tensor(POWER))\n", "model = QEPRegressionModel(train_x, train_y, likelihood)\n", "\n", "if torch.cuda.is_available():\n", " model = model.cuda()\n", " likelihood = likelihood.cuda()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Training the model" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "scrolled": true }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "21c7d486ed024a7e99f7af92ffda0e9a", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Train: 0%| | 0/100 [00:00