Normal-inverse Gamma Distribution

In probability theory and statistics, the normal-inverse-gamma distribution (or Gaussian-inverse-gamma distribution) is a four-parameter family of multivariate continuous probability distributions. It is the conjugate prior of a normal distribution with unknown mean and variance.

Definition

Suppose

:$x \mid \sigma^2, \mu, \lambda\sim \mathrm(\mu,\sigma^2 / \lambda) \,\!$
has a normal distribution with mean $\mu$ and variance $\sigma^2 / \lambda$, where

:$\sigma^2\mid\alpha, \beta \sim \Gamma^(\alpha,\beta) \!$
has an inverse gamma distribution. Then $(x,\sigma^2)$
has a normal-inverse-gamma distribution, denoted as
:$(x,\sigma^2) \sim \text\Gamma^(\mu,\lambda,\alpha,\beta) \! .$

($\text$ is also used instead of $\text\Gamma^.$)

The normal-inverse-Wishart distribution is a generalization of the normal-inverse-gamma distribution that is defined over multivariate random variables.

Characterization

Probability density function

: $f(x,\sigma^2\mid\mu,\lambda,\alpha,\beta) = \frac \, \frac \, \left( \frac \right)^ \exp \left( -\frac \right)$

For the multivariate form where $\mathbf$ is a $k \times 1$ random vector,

: $f(\mathbf,\sigma^2\mid\mu,\mathbf^,\alpha,\beta) = , \mathbf, ^ \, \frac \, \left( \frac \right)^ \exp \left( -\frac \right).$

where $, \mathbf,$ is the determinant of the $k \times k$ matrix $\mathbf$. Note how this last equation reduces to the first form if $k = 1$ so that $\mathbf, \mathbf, \boldsymbol$ are scalars.

Alternative parameterization

It is also possible to let $\gamma = 1 / \lambda$ in which case the pdf becomes

: $f(x,\sigma^2\mid\mu,\gamma,\alpha,\beta) = \frac \, \frac \, \left( \frac \right)^ \exp \left( -\frac \right)$

In the multivariate form, the corresponding change would be to regard the covariance matrix $\mathbf$ instead of its inverse $\mathbf^$ as a parameter.

Cumulative distribution function

: $F(x,\sigma^2\mid\mu,\lambda,\alpha,\beta) = \frac$

Properties

Marginal distributions

Given $(x,\sigma^2) \sim \text\Gamma^(\mu,\lambda,\alpha,\beta) \! .$ as above, $\sigma^2$ by itself follows an inverse gamma distribution:

:$\sigma^2 \sim \Gamma^(\alpha,\beta) \!$

while $\sqrt (x - \mu)$ follows a t distribution with $2 \alpha$ degrees of freedom.

In the multivariate case, the marginal distribution of $\mathbf$ is a multivariate t distribution:

:$\mathbf \sim t_(\boldsymbol, \frac \mathbf^) \!$

Summation

Scaling

Suppose

:$(x,\sigma^2) \sim \text\Gamma^(\mu,\lambda,\alpha,\beta) \! .$

Then for $c>0$,
:$(cx,c\sigma^2) \sim \text\Gamma^(c\mu,\lambda/c,\alpha,c\beta) \! .$

Proof: To prove this let $(x,\sigma^2) \sim \text\Gamma^(\mu,\lambda,\alpha,\beta)$ and fix $c>0$. Defining $Y=(Y_1,Y_2)=(cx,c \sigma^2)$, observe that the PDF of the random variable $Y$ evaluated at $(y_1,y_2)$ is given by $1/c^2$ times the PDF of a $\text\Gamma^(\mu,\lambda,\alpha,\beta)$ random variable evaluated at $(y_1/c,y_2/c)$. Hence the PDF of $Y$ evaluated at $(y_1,y_2)$ is given by :$f_Y(y_1,y_2)=\frac \frac \, \frac \, \left( \frac \right)^ \exp \left( -\frac \right) = \frac \, \frac \, \left( \frac \right)^ \exp \left( -\frac \right).\!$

The right hand expression is the PDF for a $\text\Gamma^(c\mu,\lambda/c,\alpha,c\beta)$ random variable evaluated at $(y_1,y_2)$, which completes the proof.

Exponential family

Normal distributions form an exponential family with natural parameters $\textstyle\theta_1=\frac$, $\textstyle\theta_2=\lambda \mu$, $\textstyle\theta_3=\alpha$, and $\textstyle\theta_4=-\beta+\frac$ and sufficient statistics $\textstyle T_1=\frac$, $\textstyle T_2=\frac$, $\textstyle T_3=\log \big( \frac \big)$, and $\textstyle T_4=\frac$.

Information entropy

Kullback–Leibler divergence

Measures difference between two distributions.

Maximum likelihood estimation

Posterior distribution of the parameters

See the articles on normal-gamma distribution and conjugate prior.

Interpretation of the parameters

See the articles on normal-gamma distribution and conjugate prior.

Generating normal-inverse-gamma random variates

Generation of random variates is straightforward:
# Sample $\sigma^2$ from an inverse gamma distribution with parameters $\alpha$ and $\beta$
# Sample $x$ from a normal distribution with mean $\mu$ and variance $\sigma^2/\lambda$

Related distributions

* The normal-gamma distribution is the same distribution parameterized by precision rather than variance
* A generalization of this distribution which allows for a multivariate mean and a completely unknown positive-definite covariance matrix $\sigma^2 \mathbf$ (whereas in the multivariate inverse-gamma distribution the covariance matrix is regarded as known up to the scale factor $\sigma^2$) is the normal-inverse-Wishart distribution

See also

* Compound probability distribution

References

* Denison, David G. T. ; Holmes, Christopher C.; Mallick, Bani K.; Smith, Adrian F. M. (2002) ''Bayesian Methods for Nonlinear Classification and Regression'', Wiley.
* Koch, Karl-Rudolf (2007) ''Introduction to Bayesian Statistics'' (2nd Edition), Springer.

{{ProbDistributions, multivariate

Continuous distributions
Multivariate continuous distributions
Normal distribution