Normal-inverse-Wishart Distribution

In probability theory and statistics, the normal-inverse-Wishart distribution (or Gaussian-inverse-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate normal distribution with unknown mean and covariance matrix (the inverse of the precision matrix).Murphy, Kevin P. (2007). "Conjugate Bayesian analysis of the Gaussian distribution.

/ref>

Definition

Suppose

:$\boldsymbol\mu, \boldsymbol\mu_0,\lambda,\boldsymbol\Sigma \sim \mathcal\left(\boldsymbol\mu\Big, \boldsymbol\mu_0,\frac\boldsymbol\Sigma\right)$
has a multivariate normal distribution with mean $\boldsymbol\mu_0$ and covariance matrix $\tfrac\boldsymbol\Sigma$, where

:$\boldsymbol\Sigma, \boldsymbol\Psi,\nu \sim \mathcal^(\boldsymbol\Sigma, \boldsymbol\Psi,\nu)$
has an inverse Wishart distribution. Then $(\boldsymbol\mu,\boldsymbol\Sigma)$
has a normal-inverse-Wishart distribution, denoted as
:$(\boldsymbol\mu,\boldsymbol\Sigma) \sim \mathrm(\boldsymbol\mu_0,\lambda,\boldsymbol\Psi,\nu) .$

Characterization

Probability density function

: $f(\boldsymbol\mu,\boldsymbol\Sigma, \boldsymbol\mu_0,\lambda,\boldsymbol\Psi,\nu) = \mathcal\left(\boldsymbol\mu\Big, \boldsymbol\mu_0,\frac\boldsymbol\Sigma\right) \mathcal^(\boldsymbol\Sigma, \boldsymbol\Psi,\nu)$

The full version of the PDF is as follows:

$f(\boldsymbol,\boldsymbol , \boldsymbol,\gamma,\boldsymbol,\alpha ) =\frac\text\left\$

Here \Gamma_D cdot/math> is the multivariate gamma function and $Tr(\boldsymbol)$ is the Trace of the given matrix.

Properties

Scaling

Marginal distributions

By construction, the marginal distribution over $\boldsymbol\Sigma$ is an inverse Wishart distribution, and the conditional distribution over $\boldsymbol\mu$ given $\boldsymbol\Sigma$ is a multivariate normal distribution. The marginal distribution over $\boldsymbol\mu$ is a multivariate t-distribution.

Posterior distribution of the parameters

Suppose the sampling density is a multivariate normal distribution

:$\boldsymbol, \boldsymbol\mu,\boldsymbol\Sigma \sim \mathcal_p(\boldsymbol\mu,\boldsymbol\Sigma)$

where $\boldsymbol$ is an $n\times p$ matrix and $\boldsymbol$ (of length $p$) is row $i$ of the matrix .

With the mean and covariance matrix of the sampling distribution is unknown, we can place a Normal-Inverse-Wishart prior on the mean and covariance parameters jointly

:$(\boldsymbol\mu,\boldsymbol\Sigma) \sim \mathrm(\boldsymbol\mu_0,\lambda,\boldsymbol\Psi,\nu).$

The resulting posterior distribution for the mean and covariance matrix will also be a Normal-Inverse-Wishart

:$(\boldsymbol\mu,\boldsymbol\Sigma, y) \sim \mathrm(\boldsymbol\mu_n,\lambda_n,\boldsymbol\Psi_n,\nu_n),$

where

:$\boldsymbol\mu_n = \frac$

:$\lambda_n = \lambda + n$

:$\nu_n = \nu + n$

:$\boldsymbol\Psi_n = \boldsymbol +\frac (\boldsymbol)(\boldsymbol)^T ~~~\mathrm~~\boldsymbol= \sum_^ (\boldsymbol)(\boldsymbol)^T$.

To sample from the joint posterior of $(\boldsymbol\mu,\boldsymbol\Sigma)$, one simply draws samples from $\boldsymbol\Sigma, \boldsymbol y \sim \mathcal^(\boldsymbol\Psi_n,\nu_n)$, then draw $\boldsymbol\mu , \boldsymbol \sim \mathcal_p(\boldsymbol\mu_n,\boldsymbol\Sigma/\lambda_n)$. To draw from the posterior predictive of a new observation, draw $\boldsymbol\tilde, \boldsymbol \sim \mathcal_p(\boldsymbol\mu,\boldsymbol\Sigma)$ , given the already drawn values of $\boldsymbol\mu$ and $\boldsymbol\Sigma$.Gelman, Andrew, et al. Bayesian data analysis. Vol. 2, p.73. Boca Raton, FL, USA: Chapman & Hall/CRC, 2014.

Generating normal-inverse-Wishart random variates

Generation of random variates is straightforward:
# Sample $\boldsymbol\Sigma$ from an inverse Wishart distribution with parameters $\boldsymbol\Psi$ and $\nu$
# Sample $\boldsymbol\mu$ from a multivariate normal distribution with mean $\boldsymbol\mu_0$ and variance $\boldsymbol \tfrac \boldsymbol\Sigma$

Related distributions

* The normal-Wishart distribution is essentially the same distribution parameterized by precision rather than variance. If $(\boldsymbol\mu,\boldsymbol\Sigma) \sim \mathrm(\boldsymbol\mu_0,\lambda,\boldsymbol\Psi,\nu)$ then $(\boldsymbol\mu,\boldsymbol\Sigma^) \sim \mathrm(\boldsymbol\mu_0,\lambda,\boldsymbol\Psi^,\nu)$ .
* The normal-inverse-gamma distribution is the one-dimensional equivalent.
* The multivariate normal distribution and inverse Wishart distribution are the component distributions out of which this distribution is made.

Notes

References

* Bishop, Christopher M. (2006). ''Pattern Recognition and Machine Learning.'' Springer Science+Business Media.
* Murphy, Kevin P. (2007). "Conjugate Bayesian analysis of the Gaussian distribution.


{{ProbDistributions, multivariate

Multivariate continuous distributions
Conjugate prior distributions
Normal distribution