HOME

TheInfoList



OR:

In
mathematics Mathematics is a field of study that discovers and organizes methods, Mathematical theory, theories and theorems that are developed and Mathematical proof, proved for the needs of empirical sciences and mathematics itself. There are many ar ...
, specifically in the theory of Markovian
stochastic process In probability theory and related fields, a stochastic () or random process is a mathematical object usually defined as a family of random variables in a probability space, where the index of the family often has the interpretation of time. Sto ...
es in
probability theory Probability theory or probability calculus is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expre ...
, the Chapman–Kolmogorov equation (CKE) is an identity relating the
joint probability distribution A joint or articulation (or articular surface) is the connection made between bones, ossicles, or other hard structures in the body which link an animal's skeletal system into a functional whole.Saladin, Ken. Anatomy & Physiology. 7th ed. McGraw- ...
s of different sets of coordinates on a stochastic process. The equation was derived independently by both the British mathematician Sydney Chapman and the Russian mathematician
Andrey Kolmogorov Andrey Nikolaevich Kolmogorov ( rus, Андре́й Никола́евич Колмого́ров, p=ɐnˈdrʲej nʲɪkɐˈlajɪvʲɪtɕ kəlmɐˈɡorəf, a=Ru-Andrey Nikolaevich Kolmogorov.ogg, 25 April 1903 – 20 October 1987) was a Soviet ...
. The CKE is prominently used in recent
variational Bayesian methods Variational Bayesian methods are a family of techniques for approximating intractable integrals arising in Bayesian inference and machine learning. They are typically used in complex statistical models consisting of observed variables (usually ...
.


Mathematical description

Suppose that is an indexed collection of
random variables A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. The term 'random variable' in its mathematical definition refers ...
, that is, a stochastic process. Let :p_(f_1,\ldots,f_n) be the joint probability density function of the values of the random variables ''f''1 to ''fn''. Then, the Chapman–Kolmogorov equation is :p_(f_1,\ldots,f_)=\int_^p_(f_1,\ldots,f_n)\,df_n i.e. a straightforward
marginalization Social exclusion or social marginalisation is the social disadvantage and relegation to the fringe of society. It is a term that has been used widely in Europe and was first used in France in the late 20th century. In the EU context, the Euro ...
over the
nuisance variable In the theory of stochastic processes In probability theory and related fields, a stochastic () or random process is a mathematical object usually defined as a family of random variables in a probability space, where the index of the family ofte ...
. (Note that nothing yet has been assumed about the temporal (or any other) ordering of the random variables—the above equation applies equally to the marginalization of any of them.)


In terms of Markov kernels

If we consider the
Markov kernel In probability theory, a Markov kernel (also known as a stochastic kernel or probability kernel) is a map that in the general theory of Markov processes plays the role that the transition matrix does in the theory of Markov processes with a finit ...
s induced by the transitions of a
Markov process In probability theory and statistics, a Markov chain or Markov process is a stochastic process describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, ...
, the Chapman-Kolmogorov equation can be seen as giving a way of composing the kernel, generalizing the way stochastic matrices compose. Given a
measurable space In mathematics, a measurable space or Borel space is a basic object in measure theory. It consists of a set and a σ-algebra, which defines the subsets that will be measured. It captures and generalises intuitive notions such as length, area, an ...
(X,\mathcal) and a Markov kernel k:(X,\mathcal)\to(X,\mathcal), the ''two-step transition'' kernel k^2:(X,\mathcal)\to(X,\mathcal) is given by : k^2(A, x) = \int_X k(A, x') \, k(dx', x) for all x\in X and A\in\mathcal. One can interpret this as a sum, over all intermediate states, of pairs of independent probabilistic transitions. More generally, given measurable spaces (X,\mathcal), (Y,\mathcal) and (Z,\mathcal), and Markov kernels k:(X,\mathcal)\to(Y,\mathcal) and h:(Y,\mathcal)\to(Z,\mathcal), we get a composite kernel h\circ k:(X,\mathcal)\to(Z,\mathcal) by : (h\circ k)(C, x) = \int_Y h(C, y)\,k(dy, x) for all x\in X and C\in\mathcal. Because of this,
Markov kernel In probability theory, a Markov kernel (also known as a stochastic kernel or probability kernel) is a map that in the general theory of Markov processes plays the role that the transition matrix does in the theory of Markov processes with a finit ...
s, like stochastic matrices, form a
category Category, plural categories, may refer to: General uses *Classification, the general act of allocating things to classes/categories Philosophy * Category of being * ''Categories'' (Aristotle) * Category (Kant) * Categories (Peirce) * Category ( ...
.


Application to time-dilated Markov chains

When the stochastic process under consideration is Markovian, the Chapman–Kolmogorov equation is equivalent to an identity on transition densities. In the Markov chain setting, one assumes that ''i''1 < ... < ''i''''n''. Then, because of the
Markov property In probability theory and statistics, the term Markov property refers to the memoryless property of a stochastic process, which means that its future evolution is independent of its history. It is named after the Russian mathematician Andrey Ma ...
, :p_(f_1,\ldots,f_n)=p_(f_1)p_(f_2\mid f_1)\cdots p_(f_n\mid f_), where the conditional probability p_(f_i\mid f_j) is the transition probability between the times i>j. So, the Chapman–Kolmogorov equation takes the form :p_(f_3\mid f_1)=\int_^\infty p_(f_3\mid f_2)p_(f_2\mid f_1) \, df_2. Informally, this says that the probability of going from state 1 to state 3 can be found from the probabilities of going from 1 to an intermediate state 2 and then from 2 to 3, by adding up over all the possible intermediate states 2. When the
probability distribution In probability theory and statistics, a probability distribution is a Function (mathematics), function that gives the probabilities of occurrence of possible events for an Experiment (probability theory), experiment. It is a mathematical descri ...
on the state space of a Markov chain is discrete and the Markov chain is homogeneous, the Chapman–Kolmogorov equations can be expressed in terms of (possibly infinite-dimensional)
matrix multiplication In mathematics, specifically in linear algebra, matrix multiplication is a binary operation that produces a matrix (mathematics), matrix from two matrices. For matrix multiplication, the number of columns in the first matrix must be equal to the n ...
, thus: :P(t+s)=P(t)P(s)\, where ''P''(''t'') is the transition matrix of jump ''t'', i.e., ''P''(''t'') is the matrix such that entry ''(i,j)'' contains the probability of the chain moving from state ''i'' to state ''j'' in ''t'' steps. As a corollary, it follows that to calculate the transition matrix of jump ''t'', it is sufficient to raise the transition matrix of jump one to the power of ''t'', that is :P(t)=P^t.\, The differential form of the Chapman–Kolmogorov equation is known as a
master equation In physics, chemistry, and related fields, master equations are used to describe the time evolution of a system that can be modeled as being in a probabilistic combination of states at any given time, and the switching between states is determi ...
.


See also

*
Fokker–Planck equation In statistical mechanics and information theory, the Fokker–Planck equation is a partial differential equation that describes the time evolution of the probability density function of the velocity of a particle under the influence of drag (physi ...
(also known as Kolmogorov forward equation) *
Kolmogorov backward equation In probability theory, Kolmogorov equations characterize continuous-time Markov processes. In particular, they describe how the probability of a continuous-time Markov process in a certain state changes over time. There are four distinct equati ...
*
Examples of Markov chains Example may refer to: * ''exempli gratia'' (e.g.), usually read out in English as "for example" * .example, reserved as a domain name that may not be installed as a top-level domain of the Internet ** example.com, example.net, example.org, a ...
*
Category of Markov kernels In mathematics, the category of Markov kernels, often denoted Stoch, is the category whose objects are measurable spaces and whose morphisms are Markov kernels. It is analogous to the category of sets and functions, but where the arrows can be ...


Citations


Further reading

* * * {{DEFAULTSORT:Chapman-Kolmogorov equation Equations Markov processes Stochastic calculus