In
mathematics
Mathematics is a field of study that discovers and organizes methods, Mathematical theory, theories and theorems that are developed and Mathematical proof, proved for the needs of empirical sciences and mathematics itself. There are many ar ...
, Laplace's method, named after
Pierre-Simon Laplace
Pierre-Simon, Marquis de Laplace (; ; 23 March 1749 – 5 March 1827) was a French polymath, a scholar whose work has been instrumental in the fields of physics, astronomy, mathematics, engineering, statistics, and philosophy. He summariz ...
, is a technique used to approximate
integral
In mathematics, an integral is the continuous analog of a Summation, sum, which is used to calculate area, areas, volume, volumes, and their generalizations. Integration, the process of computing an integral, is one of the two fundamental oper ...
s of the form
:
where
is a twice-
differentiable
In mathematics, a differentiable function of one real variable is a function whose derivative exists at each point in its domain. In other words, the graph of a differentiable function has a non- vertical tangent line at each interior point in ...
function
Function or functionality may refer to:
Computing
* Function key, a type of key on computer keyboards
* Function model, a structured representation of processes in a system
* Function object or functor or functionoid, a concept of object-orie ...
,
is a large
number
A number is a mathematical object used to count, measure, and label. The most basic examples are the natural numbers 1, 2, 3, 4, and so forth. Numbers can be represented in language with number words. More universally, individual numbers can ...
, and the endpoints
and
could be infinite. This technique was originally presented in the book by .
In
Bayesian statistics
Bayesian statistics ( or ) is a theory in the field of statistics based on the Bayesian interpretation of probability, where probability expresses a ''degree of belief'' in an event. The degree of belief may be based on prior knowledge about ...
,
Laplace's approximation
Laplace's approximation provides an analytical expression for a posterior probability distribution by fitting a Gaussian distribution with a mean equal to the MAP solution and precision equal to the observed Fisher information. The approximat ...
can refer to either approximating the
posterior normalizing constant with Laplace's method or approximating the posterior distribution with a
Gaussian
Carl Friedrich Gauss (1777–1855) is the eponym of all of the topics listed below.
There are over 100 topics all named after this German mathematician and scientist, all in the fields of mathematics, physics, and astronomy. The English eponymo ...
centered at the
maximum a posteriori estimate. Laplace approximations are used in the
integrated nested Laplace approximations
Integrated nested Laplace approximations (INLA) is a method for approximate Bayesian inference based on Laplace's method. It is designed for a class of models called latent Gaussian models (LGMs), for which it can be a fast and accurate alternativ ...
method for fast approximations of
Bayesian inference
Bayesian inference ( or ) is a method of statistical inference in which Bayes' theorem is used to calculate a probability of a hypothesis, given prior evidence, and update it as more information becomes available. Fundamentally, Bayesian infer ...
.
Concept

Let the function
have a unique
global maximum
Global may refer to:
General
*Globe, a spherical model of celestial bodies
*Earth, the third planet from the Sun
Entertainment
* ''Global'' (Paul van Dyk album), 2003
* ''Global'' (Bunji Garlin album), 2007
* ''Global'' (Humanoid album), 198 ...
at
.
is a constant here. The following two functions are considered:
:
Then,
is the global maximum of
and
as well. Hence:
:
As ''M'' increases, the ratio for
will grow exponentially, while the ratio for
does not change. Thus, significant contributions to the integral of this function will come only from points
in a
neighborhood
A neighbourhood (Commonwealth English) or neighborhood (American English) is a geographically localized community within a larger town, city, suburb or rural area, sometimes consisting of a single street and the buildings lining it. Neigh ...
of
, which can then be estimated.
General theory
To state and motivate the method, one must make several assumptions. It is assumed that
is not an endpoint of the interval of integration and that the values
cannot be very close to
unless
is close to
.
can be expanded around
by
Taylor's theorem
In calculus, Taylor's theorem gives an approximation of a k-times differentiable function around a given point by a polynomial of degree k, called the k-th-order Taylor polynomial. For a smooth function, the Taylor polynomial is the truncation a ...
,
:
where
(see:
big O notation
Big ''O'' notation is a mathematical notation that describes the asymptotic analysis, limiting behavior of a function (mathematics), function when the Argument of a function, argument tends towards a particular value or infinity. Big O is a memb ...
).
Since
has a global maximum at
, and
is not an endpoint, it is a
stationary point
In mathematics, particularly in calculus, a stationary point of a differentiable function of one variable is a point on the graph of a function, graph of the function where the function's derivative is zero. Informally, it is a point where the ...
, i.e.
. Therefore, the second-order Taylor polynomial approximating
is
:
Then, just one more step is needed to get a Gaussian distribution. Since
is a global maximum of the function
it can be stated, by definition of the
second derivative
In calculus, the second derivative, or the second-order derivative, of a function is the derivative of the derivative of . Informally, the second derivative can be phrased as "the rate of change of the rate of change"; for example, the secon ...
, that
, thus giving the relation
for
close to
. The integral can then be approximated with:
:
If
this latter integral becomes a
Gaussian integral
The Gaussian integral, also known as the Euler–Poisson integral, is the integral of the Gaussian function f(x) = e^ over the entire real line. Named after the German mathematician Carl Friedrich Gauss, the integral is
\int_^\infty e^\,dx = \s ...
if we replace the limits of integration by
and
; when
is large this creates only a small error because the exponential decays very fast away from
. Computing this Gaussian integral we obtain:
:
A generalization of this method and extension to arbitrary precision is provided by the book .
Formal statement and proof
Suppose
is a twice continuously differentiable function on
and there exists a unique point
such that:
:
Then:
:
Lower bound: Let
. Since
is continuous there exists
such that if
then
By
Taylor's Theorem
In calculus, Taylor's theorem gives an approximation of a k-times differentiable function around a given point by a polynomial of degree k, called the k-th-order Taylor polynomial. For a smooth function, the Taylor polynomial is the truncation a ...
, for any
:
Then we have the following lower bound:
:
where the last equality was obtained by a change of variables
:
Remember
so we can take the square root of its negation.
If we divide both sides of the above inequality by
:
and take the limit we get:
:
since this is true for arbitrary
we get the lower bound:
:
Note that this proof works also when
or
(or both).
Upper bound: The proof is similar to that of the lower bound but there are a few inconveniences. Again we start by picking an
but in order for the proof to work we need
small enough so that
Then, as above, by continuity of
and
Taylor's Theorem
In calculus, Taylor's theorem gives an approximation of a k-times differentiable function around a given point by a polynomial of degree k, called the k-th-order Taylor polynomial. For a smooth function, the Taylor polynomial is the truncation a ...
we can find
so that if
, then
:
Lastly, by our assumptions (assuming
are finite) there exists an
such that if
, then
.
Then we can calculate the following upper bound:
:
If we divide both sides of the above inequality by
:
and take the limit we get:
:
Since
is arbitrary we get the upper bound:
:
And combining this with the lower bound gives the result.
Note that the above proof obviously fails when
or
(or both). To deal with these cases, we need some extra assumptions. A sufficient (not necessary) assumption is that for
:
and that the number
as above exists (note that this must be an assumption in the case when the interval