In
mathematics
Mathematics is a field of study that discovers and organizes methods, Mathematical theory, theories and theorems that are developed and Mathematical proof, proved for the needs of empirical sciences and mathematics itself. There are many ar ...
, specifically in the theory of Markovian
stochastic process
In probability theory and related fields, a stochastic () or random process is a mathematical object usually defined as a family of random variables in a probability space, where the index of the family often has the interpretation of time. Sto ...
es in
probability theory
Probability theory or probability calculus is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expre ...
, the Chapman–Kolmogorov equation (CKE) is an identity relating the
joint probability distribution
A joint or articulation (or articular surface) is the connection made between bones, ossicles, or other hard structures in the body which link an animal's skeletal system into a functional whole.Saladin, Ken. Anatomy & Physiology. 7th ed. McGraw- ...
s of different sets of coordinates on a stochastic process. The equation was derived independently by both the British mathematician
Sydney Chapman and the Russian mathematician
Andrey Kolmogorov
Andrey Nikolaevich Kolmogorov ( rus, Андре́й Никола́евич Колмого́ров, p=ɐnˈdrʲej nʲɪkɐˈlajɪvʲɪtɕ kəlmɐˈɡorəf, a=Ru-Andrey Nikolaevich Kolmogorov.ogg, 25 April 1903 – 20 October 1987) was a Soviet ...
. The CKE is prominently used in recent
variational Bayesian methods
Variational Bayesian methods are a family of techniques for approximating intractable integrals arising in Bayesian inference and machine learning. They are typically used in complex statistical models consisting of observed variables (usually ...
.
Mathematical description
Suppose that is an indexed collection of
random variables
A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. The term 'random variable' in its mathematical definition refers ...
, that is, a stochastic process. Let
:
be the joint probability density function of the values of the random variables ''f''
1 to ''f
n''. Then, the Chapman–Kolmogorov equation is
:
i.e. a straightforward
marginalization
Social exclusion or social marginalisation is the social disadvantage and relegation to the fringe of society. It is a term that has been used widely in Europe and was first used in France in the late 20th century. In the EU context, the Euro ...
over the
nuisance variable In the theory of stochastic processes
In probability theory and related fields, a stochastic () or random process is a mathematical object usually defined as a family of random variables in a probability space, where the index of the family ofte ...
.
(Note that nothing yet has been assumed about the temporal (or any other) ordering of the random variables—the above equation applies equally to the marginalization of any of them.)
In terms of Markov kernels
If we consider the
Markov kernel
In probability theory, a Markov kernel (also known as a stochastic kernel or probability kernel) is a map that in the general theory of Markov processes plays the role that the transition matrix does in the theory of Markov processes with a finit ...
s induced by the transitions of a
Markov process
In probability theory and statistics, a Markov chain or Markov process is a stochastic process describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, ...
, the Chapman-Kolmogorov equation can be seen as giving a way of composing the kernel, generalizing the
way stochastic matrices compose. Given a
measurable space
In mathematics, a measurable space or Borel space is a basic object in measure theory. It consists of a set and a σ-algebra, which defines the subsets that will be measured.
It captures and generalises intuitive notions such as length, area, an ...
and a Markov kernel
, the ''two-step transition'' kernel
is given by
:
for all
and
.
One can interpret this as a sum, over all intermediate states, of pairs of independent probabilistic transitions.
More generally, given measurable spaces
,
and
, and Markov kernels
and
, we get a composite kernel
by
:
for all
and
.
Because of this,
Markov kernel
In probability theory, a Markov kernel (also known as a stochastic kernel or probability kernel) is a map that in the general theory of Markov processes plays the role that the transition matrix does in the theory of Markov processes with a finit ...
s, like
stochastic matrices, form a
category
Category, plural categories, may refer to:
General uses
*Classification, the general act of allocating things to classes/categories Philosophy
* Category of being
* ''Categories'' (Aristotle)
* Category (Kant)
* Categories (Peirce)
* Category ( ...
.
Application to time-dilated Markov chains
When the stochastic process under consideration is
Markovian, the Chapman–Kolmogorov equation is equivalent to an identity on transition densities. In the Markov chain setting, one assumes that ''i''
1 < ... < ''i''
''n''. Then, because of the
Markov property
In probability theory and statistics, the term Markov property refers to the memoryless property of a stochastic process, which means that its future evolution is independent of its history. It is named after the Russian mathematician Andrey Ma ...
,
:
where the conditional probability
is the
transition probability between the times
. So, the Chapman–Kolmogorov equation takes the form
:
Informally, this says that the probability of going from state 1 to state 3 can be found from the probabilities of going from 1 to an intermediate state 2 and then from 2 to 3, by adding up over all the possible intermediate states 2.
When the
probability distribution
In probability theory and statistics, a probability distribution is a Function (mathematics), function that gives the probabilities of occurrence of possible events for an Experiment (probability theory), experiment. It is a mathematical descri ...
on the state space of a Markov chain is discrete and the Markov chain is homogeneous, the Chapman–Kolmogorov equations can be expressed in terms of (possibly infinite-dimensional)
matrix multiplication
In mathematics, specifically in linear algebra, matrix multiplication is a binary operation that produces a matrix (mathematics), matrix from two matrices. For matrix multiplication, the number of columns in the first matrix must be equal to the n ...
, thus:
:
where ''P''(''t'') is the transition matrix of jump ''t'', i.e., ''P''(''t'') is the matrix such that entry ''(i,j)'' contains the probability of the chain moving from state ''i'' to state ''j'' in ''t'' steps.
As a corollary, it follows that to calculate the transition matrix of jump ''t'', it is sufficient to raise the transition matrix of jump one to the power of ''t'', that is
:
The differential form of the Chapman–Kolmogorov equation is known as a
master equation
In physics, chemistry, and related fields, master equations are used to describe the time evolution of a system that can be modeled as being in a probabilistic combination of states at any given time, and the switching between states is determi ...
.
See also
*
Fokker–Planck equation
In statistical mechanics and information theory, the Fokker–Planck equation is a partial differential equation that describes the time evolution of the probability density function of the velocity of a particle under the influence of drag (physi ...
(also known as Kolmogorov forward equation)
*
Kolmogorov backward equation
In probability theory, Kolmogorov equations characterize continuous-time Markov processes. In particular, they describe how the probability of a continuous-time Markov process in a certain state changes over time. There are four distinct equati ...
*
Examples of Markov chains
Example may refer to:
* ''exempli gratia'' (e.g.), usually read out in English as "for example"
* .example, reserved as a domain name that may not be installed as a top-level domain of the Internet
** example.com, example.net, example.org, a ...
*
Category of Markov kernels
In mathematics, the category of Markov kernels, often denoted Stoch, is the category whose objects are measurable spaces and whose morphisms are Markov kernels.
It is analogous to the category of sets and functions, but where the arrows can be ...
Citations
Further reading
*
*
*
{{DEFAULTSORT:Chapman-Kolmogorov equation
Equations
Markov processes
Stochastic calculus