qpytorch.distributions¶
QPyTorch distribution objects are essentially the same as GPyTorch distribution objects excepted those highlighted. For the most part, QPyTorch relies on torch’s distribution library. However, we offer two custom distributions.
We implement a custom MultivariateQExponential that accepts
LinearOperator objects for covariance matrices. This allows us to use custom
linear algebra operations, which makes this more efficient.
In addition, we implement a MultitaskMultivariateQExponential which
can be used with multi-output Q-exponential process models.
Note
If Pyro is available, all GPyTorch distribution objects inherit Pyro’s distribution methods as well.
Distribution¶
Delta¶
- class qpytorch.distributions.Delta(v, log_density=0.0, event_dim=0, validate_args=None)[source]¶
(Borrowed from Pyro.) Degenerate discrete distribution (a single point).
Discrete distribution that assigns probability one to the single element in its support. Delta distribution parameterized by a random choice should not be used with MCMC based inference, as doing so produces incorrect results.
- Parameters:
v (torch.Tensor) – The single support element.
log_density (torch.Tensor) – An optional density for this Delta. This is useful to keep the class of Delta distributions closed under differentiable transformation.
event_dim (int) – Optional event dimension, defaults to zero.
MultivariateNormal¶
- class qpytorch.distributions.MultivariateNormal(mean, covariance_matrix, validate_args=False)[source]¶
Constructs a multivariate normal random variable, based on mean and covariance. Can be multivariate, or a batch of multivariate normals
Passing a vector mean corresponds to a multivariate normal. Passing a matrix mean corresponds to a batch of multivariate normals.
- Parameters:
- Variables:
base_sample_shape (torch.Size) – The shape of a base sample (without batching) that is used to generate a single sample.
covariance_matrix (torch.Tensor) – The covariance matrix, represented as a dense
torch.Tensorlazy_covariance_matrix (LinearOperator) – The covariance matrix, represented as a
LinearOperator.mean (torch.Tensor) – The mean.
stddev (torch.Tensor) – The standard deviation.
variance (torch.Tensor) – The variance.
- __getitem__(idx)[source]¶
Constructs a new MultivariateNormal that represents a random variable modified by an indexing operation.
The mean and covariance matrix arguments are indexed accordingly.
- Parameters:
idx – Index to apply to the mean. The covariance matrix is indexed accordingly.
- Return type:
gpytorch.distributions.multivariate_normal.MultivariateNormal
- add_jitter(noise=0.0001)[source]¶
Adds a small constant diagonal to the MVN covariance matrix for numerical stability.
- Parameters:
noise (float) – The size of the constant diagonal.
- Return type:
gpytorch.distributions.multivariate_normal.MultivariateNormal
- confidence_region()[source]¶
Returns 2 standard deviations above and below the mean.
- Return type:
- Returns:
Pair of tensors of size … x N, where N is the dimensionality of the random variable. The first (second) Tensor is the lower (upper) end of the confidence region.
- expand(batch_size)[source]¶
See
torch.distributions.Distribution.expand.- Parameters:
batch_size (Size) –
- Return type:
gpytorch.distributions.multivariate_normal.MultivariateNormal
- get_base_samples(sample_shape=torch.Size([]))[source]¶
Returns i.i.d. standard Normal samples to be used with
MultivariateNormal.rsample(base_samples=base_samples).
- rsample(sample_shape=torch.Size([]), base_samples=None)[source]¶
Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched.
For the MultivariateNormal distribution, this is accomplished through:
\[\boldsymbol \mu + \mathbf L \boldsymbol \epsilon\]where \(\boldsymbol \mu \in \mathcal R^N\) is the MVN mean, \(\mathbf L \in \mathcal R^{N \times N}\) is a “root” of the covariance matrix \(\mathbf K\) (i.e. \(\mathbf L \mathbf L^\top = \mathbf K\)), and \(\boldsymbol \epsilon \in \mathcal R^N\) is a vector of (approximately) i.i.d. standard Normal random variables.
- Parameters:
- Return type:
- Returns:
A *sample_shape x *batch_shape x N tensor of i.i.d. reparameterized samples.
- sample(sample_shape=torch.Size([]), base_samples=None)[source]¶
Generates a sample_shape shaped sample or sample_shape shaped batch of samples if the distribution parameters are batched.
Note that these samples are not reparameterized and therefore cannot be backpropagated through.
- Parameters:
- Return type:
- Returns:
A *sample_shape x *batch_shape x N tensor of i.i.d. samples.
- to_data_independent_dist()[source]¶
Convert a … x N MVN distribution into a batch of independent Normal distributions. Essentially, this throws away all covariance information and treats all dimensions as batch dimensions.
- Return type:
- Returns:
A (data-independent) Normal distribution with batch shape *batch_shape x N.
- unsqueeze(dim)[source]¶
Constructs a new MultivariateNormal with the batch shape unsqueezed by the given dimension. For example, if self.batch_shape = torch.Size([2, 3]) and dim = 0, then the returned MultivariateNormal will have batch_shape = torch.Size([1, 2, 3]). If dim = -1, then the returned MultivariateNormal will have batch_shape = torch.Size([2, 3, 1]).
- Parameters:
dim (int) –
- Return type:
gpytorch.distributions.multivariate_normal.MultivariateNormal
MultitaskMultivariateNormal¶
- class qpytorch.distributions.MultitaskMultivariateNormal(mean, covariance_matrix, validate_args=False, interleaved=True)[source]¶
Constructs a multi-output multivariate Normal random variable, based on mean and covariance Can be multi-output multivariate, or a batch of multi-output multivariate Normal
Passing a matrix mean corresponds to a multi-output multivariate Normal Passing a matrix mean corresponds to a batch of multivariate Normals
- Parameters:
mean (torch.Tensor) – An n x t or batch b x n x t matrix of means for the MVN distribution.
covar (LinearOperator) – An … x NT x NT (batch) matrix. covariance matrix of MVN distribution.
validate_args (bool) – (default=False) If True, validate mean and covariance_matrix arguments.
interleaved (bool) – (default=True) If True, covariance matrix is interpreted as block-diagonal w.r.t. inter-task covariances for each observation. If False, it is interpreted as block-diagonal w.r.t. inter-observation covariance for each task.
- __getitem__(idx)[source]¶
Constructs a new MultivariateNormal that represents a random variable modified by an indexing operation.
The mean and covariance matrix arguments are indexed accordingly.
- Parameters:
idx (Any) – Index to apply to the mean. The covariance matrix is indexed accordingly.
- Return type:
gpytorch.distributions.multivariate_normal.MultivariateNormal
- Returns:
If indices specify a slice for samples and tasks, returns a MultitaskMultivariateNormal, else returns a MultivariateNormal.
- property base_sample_shape¶
Returns the shape of a base sample (without batching) that is used to generate a single sample.
- classmethod from_batch_mvn(batch_mvn, task_dim=-1)[source]¶
Reinterprate a batch of multivariate normal distributions as an (independent) multitask multivariate normal distribution.
- Parameters:
batch_mvn (MultivariateNormal) – The base MVN distribution. (This distribution should have at least one batch dimension).
task_dim (int) – Which batch dimension should be interpreted as the dimension for the independent tasks.
- Returns:
the independent multitask distribution
- Return type:
Example
>>> # model is a gpytorch.models.VariationalGP >>> # likelihood is a gpytorch.likelihoods.Likelihood >>> mean = torch.randn(4, 2, 3) >>> covar_factor = torch.randn(4, 2, 3, 3) >>> covar = covar_factor @ covar_factor.transpose(-1, -2) >>> mvn = gpytorch.distributions.MultivariateNormal(mean, covar) >>> print(mvn.event_shape, mvn.batch_shape) >>> # torch.Size([3]), torch.Size([4, 2]) >>> >>> mmvn = MultitaskMultivariateNormal.from_batch_mvn(mvn, task_dim=-1) >>> print(mmvn.event_shape, mmvn.batch_shape) >>> # torch.Size([3, 2]), torch.Size([4])
- classmethod from_independent_mvns(mvns)[source]¶
Convert an iterable of MVNs into a
MultitaskMultivariateNormal. The resulting distribution will havelen(mvns)tasks, and the tasks will be independent.- Parameters:
mvn (MultitaskNormal) – The base MVN distributions.
- Returns:
the independent multitask distribution
- Return type:
Example
>>> # model is a gpytorch.models.VariationalGP >>> # likelihood is a gpytorch.likelihoods.Likelihood >>> mean = torch.randn(4, 3) >>> covar_factor = torch.randn(4, 3, 3) >>> covar = covar_factor @ covar_factor.transpose(-1, -2) >>> mvn1 = gpytorch.distributions.MultivariateNormal(mean, covar) >>> >>> mean = torch.randn(4, 3) >>> covar_factor = torch.randn(4, 3, 3) >>> covar = covar_factor @ covar_factor.transpose(-1, -2) >>> mvn2 = gpytorch.distributions.MultivariateNormal(mean, covar) >>> >>> mmvn = MultitaskMultivariateNormal.from_independent_mvns([mvn1, mvn2]) >>> print(mmvn.event_shape, mmvn.batch_shape) >>> # torch.Size([3, 2]), torch.Size([4])
- classmethod from_repeated_mvn(mvn, num_tasks)[source]¶
Convert a single MVN into a
MultitaskMultivariateNormal, where each task shares the same mean and covariance.- Parameters:
mvn (MultitaskNormal) – The base MVN distribution.
num_tasks (int) – How many tasks to create.
- Returns:
the independent multitask distribution
- Return type:
Example
>>> # model is a gpytorch.models.VariationalGP >>> # likelihood is a gpytorch.likelihoods.Likelihood >>> mean = torch.randn(4, 3) >>> covar_factor = torch.randn(4, 3, 3) >>> covar = covar_factor @ covar_factor.transpose(-1, -2) >>> mvn = gpytorch.distributions.MultivariateNormal(mean, covar) >>> print(mvn.event_shape, mvn.batch_shape) >>> # torch.Size([3]), torch.Size([4]) >>> >>> mmvn = MultitaskMultivariateNormal.from_repeated_mvn(mvn, num_tasks=2) >>> print(mmvn.event_shape, mmvn.batch_shape) >>> # torch.Size([3, 2]), torch.Size([4])
- to_data_independent_dist(jitter_val=0.0001)[source]¶
Convert a multitask MVN into a batched (non-multitask) MVNs The result retains the intertask covariances, but gets rid of the inter-data covariances. The resulting distribution will have
len(mvns)tasks, and the tasks will be independent.- Returns:
the bached data-independent MVN
- Return type:
QExponential¶
- class qpytorch.distributions.QExponential(loc, scale, power=tensor(2.), validate_args=None)[source]¶
Creates a q-exponential distribution parameterized by
loc,scaleandpower, with the following density\[p(x; \mu, \sigma^2) = \frac{q}{2}(2\pi\sigma^2)^{-\frac{1}{2}} \left|\frac{x-\mu}{\sigma}\right|^{\frac{q}{2}-1} \exp\left\{-\frac{1}{2}\left|\frac{x-\mu}{\sigma}\right|^q\right\}\]Example:
>>> # xdoctest: +IGNORE_WANT("non-deterministic") >>> m = QExponential(torch.tensor([0.0]), torch.tensor([1.0])) >>> m.sample() # q-exponentially distributed with loc=0, scale=1 and power=2 tensor([ 0.1046])
MultivariateQExponential¶
- class qpytorch.distributions.MultivariateQExponential(mean, covariance_matrix, power=tensor(2.), validate_args=False)[source]¶
Constructs a multivariate q-exponential random variable, based on mean and covariance, whose density is
\[p(x; \mu, C) = \frac{q}{2} (2\pi)^{-\frac{N}{2}} |C|^{-\frac{1}{2}} r^{\left(\frac{q}{2}-1\right)\frac{N}{2}} \exp\left\{ -0.5 * r^{\frac{q}{2}} \right\}, \quad r(x) = (x - \mu)^T C^{-1} (x - \mu).\]The result can be multivariate, or a batch of multivariate q-exponentials. Passing a vector mean corresponds to a multivariate q-exponential. Passing a matrix mean corresponds to a batch of multivariate q-exponentials.
- Parameters:
- Variables:
base_sample_shape (torch.Size) – The shape of a base sample (without batching) that is used to generate a single sample.
covariance_matrix (torch.Tensor) – The covariance matrix, represented as a dense
torch.Tensorlazy_covariance_matrix (LinearOperator) – The covariance matrix, represented as a
LinearOperator.mean (torch.Tensor) – The mean.
stddev (torch.Tensor) – The standard deviation.
variance (torch.Tensor) – The variance.
- __getitem__(idx)[source]¶
Constructs a new MultivariateQExponential that represents a random variable modified by an indexing operation.
The mean and covariance matrix arguments are indexed accordingly.
- Parameters:
idx – Index to apply to the mean. The covariance matrix is indexed accordingly.
- Return type:
- add_jitter(noise=0.0001)[source]¶
Adds a small constant diagonal to the QEP covariance matrix for numerical stability.
- Parameters:
noise (float) – The size of the constant diagonal.
- Return type:
- confidence_region(rescale=False)[source]¶
Returns 2 standard deviations above and below the mean.
- Return type:
- Returns:
Pair of tensors of size … x N, where N is the dimensionality of the random variable. The first (second) Tensor is the lower (upper) end of the confidence region.
- expand(batch_size)[source]¶
See
torch.distributions.Distribution.expand.- Parameters:
batch_size (Size) –
- Return type:
- get_base_samples(sample_shape=torch.Size([]), rescale=False)[source]¶
Returns marginally identical but uncorrelated (m.i.u.) standard Q-Exponential samples to be used with
MultivariateQExponential.rsample(base_samples=base_samples).
- rsample(sample_shape=torch.Size([]), base_samples=None, **kwargs)[source]¶
Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched.
For the MultivariateQExponential distribution, this is accomplished through:
\[\boldsymbol \mu + \mathbf L \boldsymbol \epsilon\]where \(\boldsymbol \mu \in \mathcal R^N\) is the QEP mean, \(\mathbf L \in \mathcal R^{N \times N}\) is a “root” of the covariance matrix \(\mathbf K\) (i.e. \(\mathbf L \mathbf L^\top = \mathbf K\)), and \(\boldsymbol \epsilon \in \mathcal R^N\) is a vector of (approximately) m.i.u. standard Q-Exponential random variables.
- Parameters:
- Return type:
- Returns:
A *sample_shape x *batch_shape x N tensor of m.i.u. reparameterized samples.
- sample(sample_shape=torch.Size([]), base_samples=None, **kwargs)[source]¶
Generates a sample_shape shaped sample or sample_shape shaped batch of samples if the distribution parameters are batched.
Note that these samples are not reparameterized and therefore cannot be backpropagated through.
- Parameters:
- Return type:
- Returns:
A *sample_shape x *batch_shape x N tensor of m.i.u. samples.
- to_data_independent_dist()¶
Convert a … x N QEP distribution into a batch of uncorrelated Q-Exponential distributions. Essentially, this throws away all covariance information and treats all dimensions as batch dimensions.
- Return type:
- Returns:
A (data-uncorrelated) Q-Exponential distribution with batch shape *batch_shape x N.
Convert a … x N QEP distribution into a batch of uncorrelated Q-Exponential distributions. Essentially, this throws away all covariance information and treats all dimensions as batch dimensions.
- Return type:
- Returns:
A (data-uncorrelated) Q-Exponential distribution with batch shape *batch_shape x N.
- unsqueeze(dim)[source]¶
Constructs a new MultivariateQExponential with the batch shape unsqueezed by the given dimension. For example, if self.batch_shape = torch.Size([2, 3]) and dim = 0, then the returned MultivariateQExponential will have batch_shape = torch.Size([1, 2, 3]). If dim = -1, then the returned MultivariateQExponential will have batch_shape = torch.Size([2, 3, 1]).
- Parameters:
dim (int) –
- Return type:
- zero_mean_qep_samples(op, num_samples, **kwargs)[source]¶
Assumes that the LinearOpeator \(\mathbf A\) is a covariance matrix, or a batch of covariance matrices. Returns samples from a zero-mean QEP, defined by \(\mathcal Q( \mathbf 0, \mathbf A)\).
- Parameters:
num_samples (int) – Number of samples to draw.
op (LinearOperator) –
- Return type:
- Returns:
Samples from QEP \(\mathcal Q( \mathbf 0, \mathbf A)\).
MultitaskMultivariateQExponential¶
- class qpytorch.distributions.MultitaskMultivariateQExponential(mean, covariance_matrix, power=tensor(2.), validate_args=False, interleaved=True)[source]¶
Constructs a multi-output multivariate Q-Exponential random variable, based on mean and covariance Can be multi-output multivariate, or a batch of multi-output multivariate Q-Exponential
Passing a matrix mean corresponds to a multi-output multivariate Q-Exponential Passing a matrix mean corresponds to a batch of multivariate Q-Exponentials
- Parameters:
mean (torch.Tensor) – An n x t or batch b x n x t matrix of means for the QEP distribution.
covar (LinearOperator) – An … x NT x NT (batch) matrix. covariance matrix of QEP distribution.
power – (default=2.0) (scalar) power of QEP distribution.
validate_args (bool) – (default=False) If True, validate mean and covariance_matrix arguments.
interleaved (bool) – (default=True) If True, covariance matrix is interpreted as block-diagonal w.r.t. inter-task covariances for each observation. If False, it is interpreted as block-diagonal w.r.t. inter-observation covariance for each task.
- __getitem__(idx)[source]¶
Constructs a new MultivariateQExponential that represents a random variable modified by an indexing operation.
The mean and covariance matrix arguments are indexed accordingly.
- Parameters:
idx (Any) – Index to apply to the mean. The covariance matrix is indexed accordingly.
- Return type:
- Returns:
If indices specify a slice for samples and tasks, returns a MultitaskMultivariateQExponential, else returns a MultivariateQExponential.
- property base_sample_shape¶
Returns the shape of a base sample (without batching) that is used to generate a single sample.
- classmethod from_batch_qep(batch_qep, task_dim=-1)[source]¶
Reinterpret a batch of multivariate q-exponential distributions as an (uncorrelated) multitask multivariate q-exponential distribution.
- Parameters:
batch_qep (MultivariateQExponential) – The base QEP distribution. (This distribution should have at least one batch dimension).
task_dim (int) – Which batch dimension should be interpreted as the dimension for the independent tasks.
- Returns:
the uncorrelated multitask distribution
- Return type:
Example
>>> # model is a qpytorch.models.VariationalQEP >>> # likelihood is a qpytorch.likelihoods.Likelihood >>> mean = torch.randn(4, 2, 3) >>> covar_factor = torch.randn(4, 2, 3, 3) >>> covar = covar_factor @ covar_factor.transpose(-1, -2) >>> power = torch.tensor(1.0) >>> qep = qpytorch.distributions.MultivariateQExponential(mean, covar, power) >>> print(qep.event_shape, qep.batch_shape) >>> # torch.Size([3]), torch.Size([4, 2]) >>> >>> mqep = MultitaskMultivariateQExponential.from_batch_qep(qep, task_dim=-1) >>> print(mqep.event_shape, mqep.batch_shape) >>> # torch.Size([3, 2]), torch.Size([4])
- classmethod from_repeated_qep(qep, num_tasks)[source]¶
Convert a single QEP into a
MultitaskMultivariateQExponential, where each task shares the same mean and covariance.- Parameters:
qep (MultivariateQExponential) – The base QEP distribution.
num_tasks (int) – How many tasks to create.
- Returns:
the uncorrelated multitask distribution
- Return type:
Example
>>> # model is a qpytorch.models.VariationalQEP >>> # likelihood is a qpytorch.likelihoods.Likelihood >>> mean = torch.randn(4, 3) >>> covar_factor = torch.randn(4, 3, 3) >>> covar = covar_factor @ covar_factor.transpose(-1, -2) >>> qep = qpytorch.distributions.MultivariateQExponential(mean, covar) >>> print(qep.event_shape, qep.batch_shape) >>> # torch.Size([3]), torch.Size([4]) >>> >>> mqep = MultitaskMultivariateQExponential.from_repeated_qep(qep, num_tasks=2) >>> print(mqep.event_shape, mqep.batch_shape) >>> # torch.Size([3, 2]), torch.Size([4])
Convert an iterable of QEPs into a
MultitaskMultivariateQExponential. The resulting distribution will havelen(qeps)tasks, and the tasks will be uncorrelated.- Parameters:
qep (MultivariateQExponential) – The base QEP distributions.
- Returns:
the uncorrelated multitask distribution
- Return type:
Example
>>> # model is a qpytorch.models.VariationalQEP >>> # likelihood is a qpytorch.likelihoods.Likelihood >>> mean = torch.randn(4, 3) >>> covar_factor = torch.randn(4, 3, 3) >>> covar = covar_factor @ covar_factor.transpose(-1, -2) >>> power = torch.tensor(1.0) >>> qep1 = qpytorch.distributions.MultivariateQExponential(mean, covar, power) >>> >>> mean = torch.randn(4, 3) >>> covar_factor = torch.randn(4, 3, 3) >>> covar = covar_factor @ covar_factor.transpose(-1, -2) >>> qep2 = qpytorch.distributions.MultivariateQExponential(mean, covar, power) >>> >>> mqep = MultitaskMultivariateQExponential.from_uncorrelated_qeps([qep1, qep2]) >>> print(mqep.event_shape, mqep.batch_shape) >>> # torch.Size([3, 2]), torch.Size([4])
Convert a multitask QEP into a batched (non-multitask) QEPs The result retains the intertask covariances, but gets rid of the inter-data covariances. The resulting distribution will have
len(qeps)tasks, and the tasks will be uncorrelated.- Returns:
the bached data-uncorrelated QEP
- Return type:
Power¶
- class qpytorch.distributions.Power(power_init=tensor(1.), power_constraint=None, power_prior=None)[source]¶
Constructs a power parameter for the (multivariate) q-exponential distribution. See
qpytorch.distributions.QExponentialorqpytorch.distributions.MultivariateQExponentialfor description of the power parameter.Note
This object works similarly as a hyperparameter of kernel, which can be imposed with a prior and optimized over.
- Parameters:
power_init (Tensor) – initial value of power parameter of qep distribution. (Default: 1.0)
power_constraint (Optional) – Set this if you want to apply a constraint to the power parameter. (Default:
Positive.)power_prior (Optional) – Set this if you want to apply a prior to the power parameter. (Default: None.)
- Variables:
shape (torch.Size) – The dimension of the power object.
power (torch.Tensor) – The power parameter. The size/shape is the same as the power_init argument.
data (torch.Tensor) – The data of the power object in
torch.tensorformat.
Example
>>> power_init = torch.tensor(1.0) >>> power_prior = qpytorch.priors.GammaPrior(4.0, 2.0) >>> power = qpytorch.distributions.Power(power_init, power_prior=power_prior)