In
probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set o ...
and
statistics, the normal-inverse-gamma distribution (or Gaussian-inverse-gamma distribution) is a four-parameter family of multivariate continuous
probability distribution
In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomeno ...
s. It is the
conjugate prior
In Bayesian probability theory, if the posterior distribution p(\theta \mid x) is in the same probability distribution family as the prior probability distribution p(\theta), the prior and posterior are then called conjugate distributions, and t ...
of a
normal distribution
In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is
:
f(x) = \frac e^
The parameter \mu i ...
with unknown
mean
There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value ( magnitude and sign) of a given data set.
For a data set, the '' ari ...
and
variance
In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of number ...
.
Definition
Suppose
:
has a
normal distribution
In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is
:
f(x) = \frac e^
The parameter \mu i ...
with
mean
There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value ( magnitude and sign) of a given data set.
For a data set, the '' ari ...
and
variance
In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of number ...
, where
:
has an
inverse gamma distribution
In probability theory and statistics, the inverse gamma distribution is a two-parameter family of continuous probability distributions on the positive real line, which is the distribution of the reciprocal of a variable distributed according to ...
. Then
has a normal-inverse-gamma distribution, denoted as
:
(
is also used instead of
)
The
normal-inverse-Wishart distribution
In probability theory and statistics, the normal-inverse-Wishart distribution (or Gaussian-inverse-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate ...
is a generalization of the normal-inverse-gamma distribution that is defined over multivariate random variables.
Characterization
Probability density function
:
For the multivariate form where
is a
random vector,
:
where
is the
determinant
In mathematics, the determinant is a scalar value that is a function of the entries of a square matrix. It characterizes some properties of the matrix and the linear map represented by the matrix. In particular, the determinant is nonzero if ...
of the
matrix
Matrix most commonly refers to:
* ''The Matrix'' (franchise), an American media franchise
** '' The Matrix'', a 1999 science-fiction action film
** "The Matrix", a fictional setting, a virtual reality environment, within ''The Matrix'' (franchi ...
. Note how this last equation reduces to the first form if
so that
are
scalars
Scalar may refer to:
*Scalar (mathematics), an element of a field, which is used to define a vector space, usually the field of real numbers
*Scalar (physics), a physical quantity that can be described by a single element of a number field such a ...
.
Alternative parameterization
It is also possible to let
in which case the pdf becomes
:
In the multivariate form, the corresponding change would be to regard the covariance matrix
instead of its
inverse as a parameter.
Cumulative distribution function
:
Properties
Marginal distributions
Given
as above,
by itself follows an
inverse gamma distribution
In probability theory and statistics, the inverse gamma distribution is a two-parameter family of continuous probability distributions on the positive real line, which is the distribution of the reciprocal of a variable distributed according to ...
:
:
while
follows a
t distribution with
degrees of freedom.
In the multivariate case, the marginal distribution of
is a
multivariate t distribution:
:
Summation
Scaling
Suppose
:
Then for
,
:
Proof: To prove this let
and fix
. Defining
, observe that the PDF of the random variable
evaluated at
is given by
times the PDF of a
random variable evaluated at
. Hence the PDF of
evaluated at
is given by :
The right hand expression is the PDF for a
random variable evaluated at
, which completes the proof.
Exponential family
Normal distributions form an
exponential family
In probability and statistics, an exponential family is a parametric set of probability distributions of a certain form, specified below. This special form is chosen for mathematical convenience, including the enabling of the user to calculate ...
with
natural parameter
In probability and statistics, an exponential family is a parametric set of probability distributions of a certain form, specified below. This special form is chosen for mathematical convenience, including the enabling of the user to calculate ...
s
,
,
, and
and sufficient statistics
,
,
, and
.
Information entropy
Kullback–Leibler divergence
Measures difference between two distributions.
Maximum likelihood estimation
Posterior distribution of the parameters
See the articles on
normal-gamma distribution and
conjugate prior
In Bayesian probability theory, if the posterior distribution p(\theta \mid x) is in the same probability distribution family as the prior probability distribution p(\theta), the prior and posterior are then called conjugate distributions, and t ...
.
Interpretation of the parameters
See the articles on
normal-gamma distribution and
conjugate prior
In Bayesian probability theory, if the posterior distribution p(\theta \mid x) is in the same probability distribution family as the prior probability distribution p(\theta), the prior and posterior are then called conjugate distributions, and t ...
.
Generating normal-inverse-gamma random variates
Generation of random variates is straightforward:
# Sample
from an inverse gamma distribution with parameters
and
# Sample
from a normal distribution with mean
and variance
Related distributions
* The
normal-gamma distribution is the same distribution parameterized by
precision rather than
variance
In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of number ...
* A generalization of this distribution which allows for a multivariate mean and a completely unknown positive-definite covariance matrix
(whereas in the multivariate inverse-gamma distribution the covariance matrix is regarded as known up to the scale factor
) is the
normal-inverse-Wishart distribution
In probability theory and statistics, the normal-inverse-Wishart distribution (or Gaussian-inverse-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate ...
See also
*
Compound probability distribution In probability and statistics, a compound probability distribution (also known as a mixture distribution or contagious distribution) is the probability distribution that results from assuming that a random variable is distributed according to some ...
References
* Denison, David G. T. ; Holmes, Christopher C.; Mallick, Bani K.; Smith, Adrian F. M. (2002) ''Bayesian Methods for Nonlinear Classification and Regression'', Wiley.
* Koch, Karl-Rudolf (2007) ''Introduction to Bayesian Statistics'' (2nd Edition), Springer.
{{ProbDistributions, multivariate
Continuous distributions
Multivariate continuous distributions
Normal distribution