almost sure convergence

In probability theory, there exist several different notions of convergence of sequences of random variables, including ''convergence in probability'', ''convergence in distribution'', and ''almost sure convergence''. The different notions of convergence capture different properties about the sequence, with some notions of convergence being stronger than others. For example, convergence in distribution tells us about the limit distribution of a sequence of random variables. This is a weaker notion than convergence in probability, which tells us about the value a random variable will take, rather than just the distribution.

The concept is important in probability theory, and its applications to statistics and stochastic processes. The same concepts are known in more general mathematics as stochastic convergence and they formalize the idea that certain properties of a sequence of essentially random or unpredictable events can sometimes be expected to settle down into a behavior that is essentially unchanging when items far enough into the sequence are studied. The different possible notions of convergence relate to how such a behavior can be characterized: two readily understood behaviors are that the sequence eventually takes a constant value, and that values in the sequence continue to change but can be described by an unchanging probability distribution.

Background

"Stochastic convergence" formalizes the idea that a sequence of essentially random or unpredictable events can sometimes be expected to settle into a pattern. The pattern may for instance be
*Convergence in the classical sense to a fixed value, perhaps itself coming from a random event
*An increasing similarity of outcomes to what a purely deterministic function would produce
*An increasing preference towards a certain outcome
*An increasing "aversion" against straying far away from a certain outcome
*That the probability distribution describing the next outcome may grow increasingly similar to a certain distribution

Some less obvious, more theoretical patterns could be
*That the series formed by calculating the expected value of the outcome's distance from a particular value may converge to 0
*That the variance of the random variable describing the next event grows smaller and smaller.
These other types of patterns that may arise are reflected in the different types of stochastic convergence that have been studied.

While the above discussion has related to the convergence of a single series to a limiting value, the notion of the convergence of two series towards each other is also important, but this is easily handled by studying the sequence defined as either the difference or the ratio of the two series.

For example, if the average of ''n'' independent random variables $Y_i, \ i = 1,\dots,n$, all having the same finite mean and variance, is given by

:$X_n = \frac\sum_^n Y_i\,,$

then as $n$ tends to infinity, $X_n$ converges ''in probability'' (see below) to the common mean, $\mu$, of the random variables $Y_i$. This result is known as the weak law of large numbers. Other forms of convergence are important in other useful theorems, including the central limit theorem.

Throughout the following, we assume that $(X_n)$ is a sequence of random variables, and $X$ is a random variable, and all of them are defined on the same probability space $(\Omega, \mathcal, \mathbb )$.

Convergence in distribution

Loosely, with this mode of convergence, we increasingly expect to see the next outcome in a sequence of random experiments becoming better and better modeled by a given probability distribution. More precisely, the distribution of the associated random variable in the sequence becomes arbitrarily close to a specified fixed distribution.

Convergence in distribution is the weakest form of convergence typically discussed, since it is implied by all other types of convergence mentioned in this article. However, convergence in distribution is very frequently used in practice; most often it arises from application of the central limit theorem.

Definition

A sequence $X_1, X_2, \ldots$ of real-valued random variables, with cumulative distribution functions $F_1, F_2, \ldots$, is said to converge in distribution, or converge weakly, or converge in law to a random variable with cumulative distribution function if

: $\lim_ F_n(x) = F(x),$

for every number $x \in \mathbb$ at which $F$ is continuous.

The requirement that only the continuity points of $F$ should be considered is essential. For example, if $X_n$ are distributed uniformly on intervals $\left( 0,\frac \right)$, then this sequence converges in distribution to the degenerate random variable $X=0$. Indeed, $F_n(x) = 0$ for all $n$ when $x\leq 0$, and $F_n(x) = 1$ for all $x \geq \frac$when $n > 0$. However, for this limiting random variable $F(0) = 1$, even though $F_n(0) = 0$ for all $n$. Thus the convergence of cdfs fails at the point $x=0$ where $F$ is discontinuous.

Convergence in distribution may be denoted as

where $\scriptstyle\mathcal_X$ is the law (probability distribution) of . For example, if is standard normal we can write $X_n\,\xrightarrow\,\mathcal(0,\,1)$.

For random vectors $\left\\subset \mathbb^k$ the convergence in distribution is defined similarly. We say that this sequence converges in distribution to a random -vector if
: $\lim_ \mathbb(X_n\in A) = \mathbb(X\in A)$
for every $A\subset \mathbb^k$ which is a continuity set of .

The definition of convergence in distribution may be extended from random vectors to more general random elements in arbitrary metric spaces, and even to the “random variables” which are not measurable — a situation which occurs for example in the study of empirical processes. This is the “weak convergence of laws without laws being defined” — except asymptotically.

In this case the term weak convergence is preferable (see weak convergence of measures), and we say that a sequence of random elements converges weakly to (denoted as ) if
: $\mathbb^*h(X_n) \to \mathbb\,h(X)$
for all continuous bounded functions . Here E* denotes the ''outer expectation'', that is the expectation of a “smallest measurable function that dominates ”.

Properties

* Since $F(a) = \mathbb(X \le a)$, the convergence in distribution means that the probability for to be in a given range is approximately equal to the probability that the value of is in that range, provided is sufficiently large.
*In general, convergence in distribution does not imply that the sequence of corresponding probability density functions will also converge. As an example one may consider random variables with densities . These random variables converge in distribution to a uniform ''U''(0, 1), whereas their densities do not converge at all.
** However, according to ''Scheffé’s theorem'', convergence of the probability density functions implies convergence in distribution.
* The portmanteau lemma provides several equivalent definitions of convergence in distribution. Although these definitions are less intuitive, they are used to prove a number of statistical theorems. The lemma states that converges in distribution to if and only if any of the following statements are true:
** $\mathbb(X_n \le x) \to \mathbb(X \le x)$ for all continuity points of $x\mapsto \mathbb(X \le x)$;
** $\mathbbf(X_n) \to \mathbbf(X)$ for all bounded, continuous functions $f$ (where $\mathbb$ denotes the expected value operator);
** $\mathbbf(X_n) \to \mathbbf(X)$ for all bounded, Lipschitz functions $f$;
** $\lim\inf \mathbbf(X_n) \ge \mathbbf(X)$ for all nonnegative, continuous functions $f$;
** $\lim\inf \mathbb(X_n \in G) \ge \mathbb(X \in G)$ for every open set $G$;
** $\lim\sup \mathbb(X_n \in F) \le \mathbb(X \in F)$ for every closed set $F$;
** $\mathbb(X_n \in B) \to \mathbb(X \in B)$ for all continuity sets $B$ of random variable $X$;
** $\limsup \mathbbf(X_n) \le \mathbbf(X)$ for every upper semi-continuous function $f$ bounded above;
** $\liminf \mathbbf(X_n) \ge \mathbbf(X)$ for every lower semi-continuous function $f$ bounded below.
* The continuous mapping theorem states that for a continuous function , if the sequence converges in distribution to , then converges in distribution to .
** Note however that convergence in distribution of to and to does in general ''not'' imply convergence in distribution of to or of to .
* Lévy’s continuity theorem: The sequence converges in distribution to if and only if the sequence of corresponding characteristic functions converges pointwise to the characteristic function of .
* Convergence in distribution is metrizable by the Lévy–Prokhorov metric.
* A natural link to convergence in distribution is the Skorokhod's representation theorem.

Convergence in probability

The basic idea behind this type of convergence is that the probability of an “unusual” outcome becomes smaller and smaller as the sequence progresses.

The concept of convergence in probability is used very often in statistics. For example, an estimator is called consistent if it converges in probability to the quantity being estimated. Convergence in probability is also the type of convergence established by the weak law of large numbers.

Definition

A sequence of random variables converges in probability towards the random variable ''X'' if for all ''ε'' > 0

: $\lim_\mathbb\big(, X_n-X, > \varepsilon\big) = 0.$

More explicitly, let ''P''_''n''(''ε'') be the probability that ''X''_''n'' is outside the ball of radius ''ε'' centered at ''X''. Then is said to converge in probability to ''X'' if for any and any ''δ'' > 0 there exists a number ''N'' (which may depend on ''ε'' and ''δ'') such that for all ''n'' ≥ ''N'', ''P''_''n''(''ε'') < ''δ'' (the definition of limit).

Notice that for the condition to be satisfied, it is not possible that for each ''n'' the random variables ''X'' and ''X''_''n'' are independent (and thus convergence in probability is a condition on the joint cdf's, as opposed to convergence in distribution, which is a condition on the individual cdf's), unless ''X'' is deterministic like for the weak law of large numbers. At the same time, the case of a deterministic ''X'' cannot, whenever the deterministic value is a discontinuity point (not isolated), be handled by convergence in distribution, where discontinuity points have to be explicitly excluded.

Convergence in probability is denoted by adding the letter ''p'' over an arrow indicating convergence, or using the "plim" probability limit operator:

For random elements on a separable metric space , convergence in probability is defined similarly by
: $\forall\varepsilon>0, \mathbb\big(d(X_n,X)\geq\varepsilon\big) \to 0.$

Properties

* Convergence in probability implies convergence in distribution. roof/sup>
* In the opposite direction, convergence in distribution implies convergence in probability when the limiting random variable ''X'' is a constant. roof/sup>
* Convergence in probability does not imply almost sure convergence. roof/sup>
* The continuous mapping theorem states that for every continuous function $g$, if $X_n \xrightarrow X$, then also 
* Convergence in probability defines a topology on the space of random variables over a fixed probability space. This topology is metrizable by the '' Ky Fan metric'': $d(X,Y) = \inf\!\big\$ or alternately by this metric d(X,Y)=\mathbb E\left X-Y, , 1)\right

Counterexamples

Not every sequence of random variables which converges to another random variable in distribution also converges in probability to that random variable. As an example, consider a sequence of standard normal random variables $X_n$ and a second sequence $Y_n = (-1)^nX_n$. Notice that the distribution of $Y_n$ is equal to the distribution of $X_n$ for all $n$, but:
$P(, X_n - Y_n, \geq \epsilon) = P(, X_n, \cdot, (1 - (-1)^n), \geq \epsilon)$

which does not converge to $0$. So we do not have convergence in probability.

Almost sure convergence

This is the type of stochastic convergence that is most similar to pointwise convergence known from elementary real analysis.

Definition

To say that the sequence converges almost surely or almost everywhere or with probability 1 or strongly towards ''X'' means that
$\mathbb\!\left( \lim_\! X_n = X \right) = 1.$

This means that the values of approach the value of ''X'', in the sense that events for which does not converge to ''X'' have probability 0 (see ''Almost surely''). Using the probability space $(\Omega, \mathcal, \mathbb )$ and the concept of the random variable as a function from Ω to R, this is equivalent to the statement
$\mathbb\Bigl( \omega \in \Omega: \lim_ X_n(\omega) = X(\omega) \Bigr) = 1.$

Using the notion of the limit superior of a sequence of sets, almost sure convergence can also be defined as follows:
$\mathbb\Bigl( \limsup_ \bigl\ \Bigr) = 0 \quad\text\quad \varepsilon>0.$

Almost sure convergence is often denoted by adding the letters ''a.s.'' over an arrow indicating convergence:

For generic random elements on a metric space $(S,d)$, convergence almost surely is defined similarly:
$\mathbb\Bigl( \omega\in\Omega\colon\, d\big(X_n(\omega),X(\omega)\big)\,\underset\,0 \Bigr) = 1$

Properties

* Almost sure convergence implies convergence in probability (by Fatou's lemma), and hence implies convergence in distribution. It is the notion of convergence used in the strong law of large numbers.
* The concept of almost sure convergence does not come from a topology on the space of random variables. This means there is no topology on the space of random variables such that the almost surely convergent sequences are exactly the converging sequences with respect to that topology. In particular, there is no metric of almost sure convergence.

Counterexamples

Consider a sequence $\$ of independent random variables such that $P(X_n=1)=\frac$ and $P(X_n=0)=1-\frac$. For $0<\varepsilon<1/2$ we have $P(, X_n, \geq \varepsilon)=\frac$ which converges to $0$ hence $X_n\to 0$ in probability.

Since $\sum_P(X_n=1)\to\infty$ and the events $\$ are independent, second Borel Cantelli Lemma ensures that $P(\limsup_n \)=1$ hence the sequence $\$ does not converge to $0$ almost everywhere (in fact the set on which this sequence does not converge to $0$ has probability $1$).

Sure convergence or pointwise convergence

To say that the sequence of random variables (''X''_''n'') defined over the same probability space (i.e., a random process) converges surely or everywhere or pointwise towards ''X'' means

$\forall \omega \in \Omega \colon \ \lim_ X_n(\omega) = X(\omega),$

where Ω is the sample space of the underlying probability space over which the random variables are defined.

This is the notion of pointwise convergence of a sequence of functions extended to a sequence of random variables. (Note that random variables themselves are functions).

$\left\ = \Omega.$

Sure convergence of a random variable implies all the other kinds of convergence stated above, but there is no payoff in probability theory by using sure convergence compared to using almost sure convergence. The difference between the two only exists on sets with probability zero. This is why the concept of sure convergence of random variables is very rarely used.

Convergence in mean

Given a real number , we say that the sequence converges in the ''r''-th mean (or in the ''L^r''-norm) towards the random variable ''X'', if the -th absolute moments $\mathbb$(, ''X_n'', ^{''r ''}) and $\mathbb$(, ''X'', ^{''r ''}) of and ''X'' exist, and
: $\lim_ \mathbb\left( , X_n-X, ^r \right) = 0,$
where the operator E denotes the expected value. Convergence in -th mean tells us that the expectation of the -th power of the difference between $X_n$ and $X$ converges to zero.

This type of convergence is often denoted by adding the letter ''L^r'' over an arrow indicating convergence:

The most important cases of convergence in ''r''-th mean are:
* When converges in ''r''-th mean to ''X'' for ''r'' = 1, we say that converges in mean to ''X''.
* When converges in ''r''-th mean to ''X'' for ''r'' = 2, we say that converges in mean square (or in quadratic mean) to ''X''.

Convergence in the ''r''-th mean, for ''r'' ≥ 1, implies convergence in probability (by Markov's inequality). Furthermore, if ''r'' > ''s'' ≥ 1, convergence in ''r''-th mean implies convergence in ''s''-th mean. Hence, convergence in mean square implies convergence in mean.

Additionally,
: \overset \quad\Rightarrow\quad \lim_ \mathbb X, ^r
The converse is not necessarily true, however it is true if $\overset$ (by a more general version of Scheffé's lemma).

Properties

Provided the probability space is complete:
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ) and $X_n Y_n\xrightarrow\ XY$.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ) and $X_n Y_n\xrightarrow\ XY$.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ).
* None of the above statements are true for convergence in distribution.

The chain of implications between the various notions of convergence are noted in their respective sections. They are, using the arrow notation:

: $\begin \xrightarrow & \underset & \xrightarrow & & \\ & & \Downarrow & & \\ \xrightarrow & \Rightarrow & \xrightarrow & \Rightarrow & \xrightarrow \end$

These properties, together with a number of other special cases, are summarized in the following list:

* Almost sure convergence implies convergence in probability: <sup>roof/sup>
*:$X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in probability implies there exists a sub-sequence $(n_k)$ which almost surely converges:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_\ \xrightarrow\ X$
* Convergence in probability implies convergence in distribution: roof/sup>
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in ''r''-th order mean implies convergence in probability:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in ''r''-th order mean implies convergence in lower order mean, assuming that both orders are greater than or equal to one:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X,$ provided ''r'' ≥ ''s'' ≥ 1.
* If ''X''_''n'' converges in distribution to a constant ''c'', then ''X''_''n'' converges in probability to ''c'': roof/sup>
*: $X_n\ \xrightarrow\ c \quad\Rightarrow\quad X_n\ \xrightarrow\ c,$ provided ''c'' is a constant.
* If converges in distribution to ''X'' and the difference between ''X_n'' and ''Y_n'' converges in probability to zero, then ''Y_n'' also converges in distribution to ''X'': roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ , X_n-Y_n, \ \xrightarrow\ 0\ \quad\Rightarrow\quad Y_n\ \xrightarrow\ X$
* If converges in distribution to ''X'' and ''Y_n'' converges in distribution to a constant ''c'', then the joint vector converges in distribution to : roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ c\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,c)$ provided ''c'' is a constant.
*:Note that the condition that converges to a constant is important, if it were to converge to a random variable ''Y'' then we wouldn't be able to conclude that converges to .
* If ''X_n'' converges in probability to ''X'' and ''Y_n'' converges in probability to ''Y'', then the joint vector converges in probability to : roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ Y\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,Y)$
* If converges in probability to ''X'', and if for all ''n'' and some ''b'', then converges in ''r''th mean to ''X'' for all . In other words, if converges in probability to ''X'' and all random variables are almost surely bounded above and below, then converges to ''X'' also in any ''r''th mean.
* Almost sure representation. Usually, convergence in distribution does not imply convergence almost surely. However, for a given sequence which converges in distribution to ''X''₀ it is always possible to find a new probability space (Ω, ''F'', P) and random variables defined on it such that ''Y_n'' is equal in distribution to for each , and ''Y_n'' converges to ''Y''₀ almost surely.
* If for all ''ε'' > 0,
*::$\sum_n \mathbb \left(, X_n - X, > \varepsilon\right) < \infty,$
*:then we say that ''converges almost completely'', or ''almost in probability'' towards ''X''. When converges almost completely towards ''X'' then it also converges almost surely to ''X''. In other words, if converges in probability to ''X'' sufficiently quickly (i.e. the above sequence of tail probabilities is summable for all ), then also converges almost surely to ''X''. This is a direct implication from the Borel–Cantelli lemma.
* If is a sum of ''n'' real independent random variables:
*::$S_n = X_1+\cdots+X_n \,$
*:then converges almost surely if and only if converges in probability. The proof can be found in Page 126 (Theorem 5.3.4) of the book by Kai Lai Chung.
*:However, for a sequence of mutually independent random variables, convergence in probability does not imply almost sure convergence.
* The dominated convergence theorem gives sufficient conditions for almost sure convergence to imply ''L''1-convergence:
}</sup>

*A necessary and sufficient condition for ''L''¹ convergence is $X_n\xrightarrow X$ and the sequence (''X_n'') is uniformly integrable.
*If $X_n\ \xrightarrow\ X$, the followings are equivalent
**$X_n\ \xrightarrow\ X$,
** \mathbb X, ^r< \infty ,
**$\$ is uniformly integrable.

See also

* Proofs of convergence of random variables
* Convergence of measures
* Convergence in measure
* Continuous stochastic process: the question of continuity of a stochastic process is essentially a question of convergence, and many of the same concepts and relationships used above apply to the continuity question.
* Asymptotic distribution
* Big O in probability notation
* Skorokhod's representation theorem
* The Tweedie convergence theorem
* Slutsky's theorem
* Continuous mapping theorem

Notes

References

*
*
*
*
*
*
*
*
*
*
*
*
*
*
*

{{DEFAULTSORT:Convergence Of Random Variables
Stochastic processes
Random variables, Convergence of>X_n, ^r= \mathbb X, ^r
The converse is not necessarily true, however it is true if $\overset$ (by a more general version of Scheffé's lemma).

Properties

Provided the probability space is complete:
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ) and $X_n Y_n\xrightarrow\ XY$.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ) and $X_n Y_n\xrightarrow\ XY$.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ).
* None of the above statements are true for convergence in distribution.

The chain of implications between the various notions of convergence are noted in their respective sections. They are, using the arrow notation:

: $\begin \xrightarrow & \underset & \xrightarrow & & \\ & & \Downarrow & & \\ \xrightarrow & \Rightarrow & \xrightarrow & \Rightarrow & \xrightarrow \end$

These properties, together with a number of other special cases, are summarized in the following list:

* Almost sure convergence implies convergence in probability: <sup>roof/sup>
*:$X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in probability implies there exists a sub-sequence $(n_k)$ which almost surely converges:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_\ \xrightarrow\ X$
* Convergence in probability implies convergence in distribution: roof/sup>
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in ''r''-th order mean implies convergence in probability:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in ''r''-th order mean implies convergence in lower order mean, assuming that both orders are greater than or equal to one:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X,$ provided ''r'' ≥ ''s'' ≥ 1.
* If ''X''_''n'' converges in distribution to a constant ''c'', then ''X''_''n'' converges in probability to ''c'': roof/sup>
*: $X_n\ \xrightarrow\ c \quad\Rightarrow\quad X_n\ \xrightarrow\ c,$ provided ''c'' is a constant.
* If converges in distribution to ''X'' and the difference between ''X_n'' and ''Y_n'' converges in probability to zero, then ''Y_n'' also converges in distribution to ''X'': roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ , X_n-Y_n, \ \xrightarrow\ 0\ \quad\Rightarrow\quad Y_n\ \xrightarrow\ X$
* If converges in distribution to ''X'' and ''Y_n'' converges in distribution to a constant ''c'', then the joint vector converges in distribution to : roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ c\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,c)$ provided ''c'' is a constant.
*:Note that the condition that converges to a constant is important, if it were to converge to a random variable ''Y'' then we wouldn't be able to conclude that converges to .
* If ''X_n'' converges in probability to ''X'' and ''Y_n'' converges in probability to ''Y'', then the joint vector converges in probability to : roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ Y\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,Y)$
* If converges in probability to ''X'', and if for all ''n'' and some ''b'', then converges in ''r''th mean to ''X'' for all . In other words, if converges in probability to ''X'' and all random variables are almost surely bounded above and below, then converges to ''X'' also in any ''r''th mean.
* Almost sure representation. Usually, convergence in distribution does not imply convergence almost surely. However, for a given sequence which converges in distribution to ''X''₀ it is always possible to find a new probability space (Ω, ''F'', P) and random variables defined on it such that ''Y_n'' is equal in distribution to for each , and ''Y_n'' converges to ''Y''₀ almost surely.
* If for all ''ε'' > 0,
*::$\sum_n \mathbb \left(, X_n - X, > \varepsilon\right) < \infty,$
*:then we say that ''converges almost completely'', or ''almost in probability'' towards ''X''. When converges almost completely towards ''X'' then it also converges almost surely to ''X''. In other words, if converges in probability to ''X'' sufficiently quickly (i.e. the above sequence of tail probabilities is summable for all ), then also converges almost surely to ''X''. This is a direct implication from the Borel–Cantelli lemma.
* If is a sum of ''n'' real independent random variables:
*::$S_n = X_1+\cdots+X_n \,$
*:then converges almost surely if and only if converges in probability. The proof can be found in Page 126 (Theorem 5.3.4) of the book by Kai Lai Chung.
*:However, for a sequence of mutually independent random variables, convergence in probability does not imply almost sure convergence.
* The dominated convergence theorem gives sufficient conditions for almost sure convergence to imply ''L''1-convergence:
}</sup>

*A necessary and sufficient condition for ''L''¹ convergence is $X_n\xrightarrow X$ and the sequence (''X_n'') is uniformly integrable.
*If $X_n\ \xrightarrow\ X$, the followings are equivalent
**$X_n\ \xrightarrow\ X$,
** \mathbb X, ^r
The converse is not necessarily true, however it is true if $\overset$ (by a more general version of Scheffé's lemma).

Properties

Provided the probability space is complete:
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ) and $X_n Y_n\xrightarrow\ XY$.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ) and $X_n Y_n\xrightarrow\ XY$.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ).
* None of the above statements are true for convergence in distribution.

The chain of implications between the various notions of convergence are noted in their respective sections. They are, using the arrow notation:

: $\begin \xrightarrow & \underset & \xrightarrow & & \\ & & \Downarrow & & \\ \xrightarrow & \Rightarrow & \xrightarrow & \Rightarrow & \xrightarrow \end$

These properties, together with a number of other special cases, are summarized in the following list:

* Almost sure convergence implies convergence in probability: <sup>roof/sup>
*:$X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in probability implies there exists a sub-sequence $(n_k)$ which almost surely converges:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_\ \xrightarrow\ X$
* Convergence in probability implies convergence in distribution: roof/sup>
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in ''r''-th order mean implies convergence in probability:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in ''r''-th order mean implies convergence in lower order mean, assuming that both orders are greater than or equal to one:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X,$ provided ''r'' ≥ ''s'' ≥ 1.
* If ''X''_''n'' converges in distribution to a constant ''c'', then ''X''_''n'' converges in probability to ''c'': roof/sup>
*: $X_n\ \xrightarrow\ c \quad\Rightarrow\quad X_n\ \xrightarrow\ c,$ provided ''c'' is a constant.
* If converges in distribution to ''X'' and the difference between ''X_n'' and ''Y_n'' converges in probability to zero, then ''Y_n'' also converges in distribution to ''X'': roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ , X_n-Y_n, \ \xrightarrow\ 0\ \quad\Rightarrow\quad Y_n\ \xrightarrow\ X$
* If converges in distribution to ''X'' and ''Y_n'' converges in distribution to a constant ''c'', then the joint vector converges in distribution to : roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ c\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,c)$ provided ''c'' is a constant.
*:Note that the condition that converges to a constant is important, if it were to converge to a random variable ''Y'' then we wouldn't be able to conclude that converges to .
* If ''X_n'' converges in probability to ''X'' and ''Y_n'' converges in probability to ''Y'', then the joint vector converges in probability to : roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ Y\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,Y)$
* If converges in probability to ''X'', and if for all ''n'' and some ''b'', then converges in ''r''th mean to ''X'' for all . In other words, if converges in probability to ''X'' and all random variables are almost surely bounded above and below, then converges to ''X'' also in any ''r''th mean.
* Almost sure representation. Usually, convergence in distribution does not imply convergence almost surely. However, for a given sequence which converges in distribution to ''X''₀ it is always possible to find a new probability space (Ω, ''F'', P) and random variables defined on it such that ''Y_n'' is equal in distribution to for each , and ''Y_n'' converges to ''Y''₀ almost surely.
* If for all ''ε'' > 0,
*::$\sum_n \mathbb \left(, X_n - X, > \varepsilon\right) < \infty,$
*:then we say that ''converges almost completely'', or ''almost in probability'' towards ''X''. When converges almost completely towards ''X'' then it also converges almost surely to ''X''. In other words, if converges in probability to ''X'' sufficiently quickly (i.e. the above sequence of tail probabilities is summable for all ), then also converges almost surely to ''X''. This is a direct implication from the Borel–Cantelli lemma.
* If is a sum of ''n'' real independent random variables:
*::$S_n = X_1+\cdots+X_n \,$
*:then converges almost surely if and only if converges in probability. The proof can be found in Page 126 (Theorem 5.3.4) of the book by Kai Lai Chung.
*:However, for a sequence of mutually independent random variables, convergence in probability does not imply almost sure convergence.
* The dominated convergence theorem gives sufficient conditions for almost sure convergence to imply ''L''1-convergence:
}</sup>

*A necessary and sufficient condition for ''L''¹ convergence is $X_n\xrightarrow X$ and the sequence (''X_n'') is uniformly integrable.
*If $X_n\ \xrightarrow\ X$, the followings are equivalent
**$X_n\ \xrightarrow\ X$,
** \mathbb X_n, ^r\rightarrow \mathbb X, ^r< \infty ,
**$\$ is uniformly integrable.

See also

* Proofs of convergence of random variables
* Convergence of measures
* Convergence in measure
* Continuous stochastic process: the question of continuity of a stochastic process is essentially a question of convergence, and many of the same concepts and relationships used above apply to the continuity question.
* Asymptotic distribution
* Big O in probability notation
* Skorokhod's representation theorem
* The Tweedie convergence theorem
* Slutsky's theorem
* Continuous mapping theorem

Notes

References

*
*
*
*
*
*
*
*
*
*
*
*
*
*
*

{{DEFAULTSORT:Convergence Of Random Variables
Stochastic processes
Random variables, Convergence of>X_n, ^r\rightarrow \mathbb X, ^r< \infty ,
**$\$ is uniformly integrable.

See also

* Proofs of convergence of random variables
* Convergence of measures
* Convergence in measure
* Continuous stochastic process: the question of continuity of a stochastic process is essentially a question of convergence, and many of the same concepts and relationships used above apply to the continuity question.
* Asymptotic distribution
* Big O in probability notation
* Skorokhod's representation theorem
* The Tweedie convergence theorem
* Slutsky's theorem
* Continuous mapping theorem

Notes

References

*
*
*
*
*
*
*
*
*
*
*
*
*
*
*

{{DEFAULTSORT:Convergence Of Random Variables
Stochastic processes
Random variables, Convergence of>X_n, ^r\rightarrow \mathbb X, ^r< \infty ,
**$\$ is uniformly integrable.

See also

* Proofs of convergence of random variables
* Convergence of measures
* Convergence in measure
* Continuous stochastic process: the question of continuity of a stochastic process is essentially a question of convergence, and many of the same concepts and relationships used above apply to the continuity question.
* Asymptotic distribution
* Big O in probability notation
* Skorokhod's representation theorem
* The Tweedie convergence theorem
* Slutsky's theorem
* Continuous mapping theorem

Notes

References

*
*
*
*
*
*
*
*
*
*
*
*
*
*
*

{{DEFAULTSORT:Convergence Of Random Variables
Stochastic processes
Random variables, Convergence of>X_n, ^r= \mathbb X, ^r
The converse is not necessarily true, however it is true if $\overset$ (by a more general version of Scheffé's lemma).

Properties

Provided the probability space is complete:
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ) and $X_n Y_n\xrightarrow\ XY$.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ) and $X_n Y_n\xrightarrow\ XY$.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ).
* None of the above statements are true for convergence in distribution.

The chain of implications between the various notions of convergence are noted in their respective sections. They are, using the arrow notation:

: $\begin \xrightarrow & \underset & \xrightarrow & & \\ & & \Downarrow & & \\ \xrightarrow & \Rightarrow & \xrightarrow & \Rightarrow & \xrightarrow \end$

These properties, together with a number of other special cases, are summarized in the following list:

* Almost sure convergence implies convergence in probability: <sup>roof/sup>
*:$X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in probability implies there exists a sub-sequence $(n_k)$ which almost surely converges:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_\ \xrightarrow\ X$
* Convergence in probability implies convergence in distribution: roof/sup>
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in ''r''-th order mean implies convergence in probability:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in ''r''-th order mean implies convergence in lower order mean, assuming that both orders are greater than or equal to one:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X,$ provided ''r'' ≥ ''s'' ≥ 1.
* If ''X''_''n'' converges in distribution to a constant ''c'', then ''X''_''n'' converges in probability to ''c'': roof/sup>
*: $X_n\ \xrightarrow\ c \quad\Rightarrow\quad X_n\ \xrightarrow\ c,$ provided ''c'' is a constant.
* If converges in distribution to ''X'' and the difference between ''X_n'' and ''Y_n'' converges in probability to zero, then ''Y_n'' also converges in distribution to ''X'': roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ , X_n-Y_n, \ \xrightarrow\ 0\ \quad\Rightarrow\quad Y_n\ \xrightarrow\ X$
* If converges in distribution to ''X'' and ''Y_n'' converges in distribution to a constant ''c'', then the joint vector converges in distribution to : roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ c\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,c)$ provided ''c'' is a constant.
*:Note that the condition that converges to a constant is important, if it were to converge to a random variable ''Y'' then we wouldn't be able to conclude that converges to .
* If ''X_n'' converges in probability to ''X'' and ''Y_n'' converges in probability to ''Y'', then the joint vector converges in probability to : roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ Y\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,Y)$
* If converges in probability to ''X'', and if for all ''n'' and some ''b'', then converges in ''r''th mean to ''X'' for all . In other words, if converges in probability to ''X'' and all random variables are almost surely bounded above and below, then converges to ''X'' also in any ''r''th mean.
* Almost sure representation. Usually, convergence in distribution does not imply convergence almost surely. However, for a given sequence which converges in distribution to ''X''₀ it is always possible to find a new probability space (Ω, ''F'', P) and random variables defined on it such that ''Y_n'' is equal in distribution to for each , and ''Y_n'' converges to ''Y''₀ almost surely.
* If for all ''ε'' > 0,
*::$\sum_n \mathbb \left(, X_n - X, > \varepsilon\right) < \infty,$
*:then we say that ''converges almost completely'', or ''almost in probability'' towards ''X''. When converges almost completely towards ''X'' then it also converges almost surely to ''X''. In other words, if converges in probability to ''X'' sufficiently quickly (i.e. the above sequence of tail probabilities is summable for all ), then also converges almost surely to ''X''. This is a direct implication from the Borel–Cantelli lemma.
* If is a sum of ''n'' real independent random variables:
*::$S_n = X_1+\cdots+X_n \,$
*:then converges almost surely if and only if converges in probability. The proof can be found in Page 126 (Theorem 5.3.4) of the book by Kai Lai Chung.
*:However, for a sequence of mutually independent random variables, convergence in probability does not imply almost sure convergence.
* The dominated convergence theorem gives sufficient conditions for almost sure convergence to imply ''L''1-convergence:
}</sup>

*A necessary and sufficient condition for ''L''¹ convergence is $X_n\xrightarrow X$ and the sequence (''X_n'') is uniformly integrable.
*If $X_n\ \xrightarrow\ X$, the followings are equivalent
**$X_n\ \xrightarrow\ X$,
** \mathbb X, ^r
The converse is not necessarily true, however it is true if $\overset$ (by a more general version of Scheffé's lemma).

Properties

Provided the probability space is complete:
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ) and $X_n Y_n\xrightarrow\ XY$.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ) and $X_n Y_n\xrightarrow\ XY$.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ).
* None of the above statements are true for convergence in distribution.

The chain of implications between the various notions of convergence are noted in their respective sections. They are, using the arrow notation:

: $\begin \xrightarrow & \underset & \xrightarrow & & \\ & & \Downarrow & & \\ \xrightarrow & \Rightarrow & \xrightarrow & \Rightarrow & \xrightarrow \end$

These properties, together with a number of other special cases, are summarized in the following list:

* Almost sure convergence implies convergence in probability: <sup>roof/sup>
*:$X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in probability implies there exists a sub-sequence $(n_k)$ which almost surely converges:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_\ \xrightarrow\ X$
* Convergence in probability implies convergence in distribution: roof/sup>
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in ''r''-th order mean implies convergence in probability:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in ''r''-th order mean implies convergence in lower order mean, assuming that both orders are greater than or equal to one:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X,$ provided ''r'' ≥ ''s'' ≥ 1.
* If ''X''_''n'' converges in distribution to a constant ''c'', then ''X''_''n'' converges in probability to ''c'': roof/sup>
*: $X_n\ \xrightarrow\ c \quad\Rightarrow\quad X_n\ \xrightarrow\ c,$ provided ''c'' is a constant.
* If converges in distribution to ''X'' and the difference between ''X_n'' and ''Y_n'' converges in probability to zero, then ''Y_n'' also converges in distribution to ''X'': roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ , X_n-Y_n, \ \xrightarrow\ 0\ \quad\Rightarrow\quad Y_n\ \xrightarrow\ X$
* If converges in distribution to ''X'' and ''Y_n'' converges in distribution to a constant ''c'', then the joint vector converges in distribution to : roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ c\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,c)$ provided ''c'' is a constant.
*:Note that the condition that converges to a constant is important, if it were to converge to a random variable ''Y'' then we wouldn't be able to conclude that converges to .
* If ''X_n'' converges in probability to ''X'' and ''Y_n'' converges in probability to ''Y'', then the joint vector converges in probability to : roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ Y\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,Y)$
* If converges in probability to ''X'', and if for all ''n'' and some ''b'', then converges in ''r''th mean to ''X'' for all . In other words, if converges in probability to ''X'' and all random variables are almost surely bounded above and below, then converges to ''X'' also in any ''r''th mean.
* Almost sure representation. Usually, convergence in distribution does not imply convergence almost surely. However, for a given sequence which converges in distribution to ''X''₀ it is always possible to find a new probability space (Ω, ''F'', P) and random variables defined on it such that ''Y_n'' is equal in distribution to for each , and ''Y_n'' converges to ''Y''₀ almost surely.
* If for all ''ε'' > 0,
*::$\sum_n \mathbb \left(, X_n - X, > \varepsilon\right) < \infty,$
*:then we say that ''converges almost completely'', or ''almost in probability'' towards ''X''. When converges almost completely towards ''X'' then it also converges almost surely to ''X''. In other words, if converges in probability to ''X'' sufficiently quickly (i.e. the above sequence of tail probabilities is summable for all ), then also converges almost surely to ''X''. This is a direct implication from the Borel–Cantelli lemma.
* If is a sum of ''n'' real independent random variables:
*::$S_n = X_1+\cdots+X_n \,$
*:then converges almost surely if and only if converges in probability. The proof can be found in Page 126 (Theorem 5.3.4) of the book by Kai Lai Chung.
*:However, for a sequence of mutually independent random variables, convergence in probability does not imply almost sure convergence.
* The dominated convergence theorem gives sufficient conditions for almost sure convergence to imply ''L''1-convergence:
}</sup>

*A necessary and sufficient condition for ''L''¹ convergence is $X_n\xrightarrow X$ and the sequence (''X_n'') is uniformly integrable.
*If $X_n\ \xrightarrow\ X$, the followings are equivalent
**$X_n\ \xrightarrow\ X$,
** \mathbb X, ^r< \infty ,
**$\$ is uniformly integrable.

See also

* Proofs of convergence of random variables
* Convergence of measures
* Convergence in measure
* Continuous stochastic process: the question of continuity of a stochastic process is essentially a question of convergence, and many of the same concepts and relationships used above apply to the continuity question.
* Asymptotic distribution
* Big O in probability notation
* Skorokhod's representation theorem
* The Tweedie convergence theorem
* Slutsky's theorem
* Continuous mapping theorem

Notes

References

*
*
*
*
*
*
*
*
*
*
*
*
*
*
*

{{DEFAULTSORT:Convergence Of Random Variables
Stochastic processes
Random variables, Convergence of>X_n, ^r= \mathbb X, ^r
The converse is not necessarily true, however it is true if $\overset$ (by a more general version of Scheffé's lemma).

Properties

Provided the probability space is complete:
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ) and $X_n Y_n\xrightarrow\ XY$.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ) and $X_n Y_n\xrightarrow\ XY$.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ).
* None of the above statements are true for convergence in distribution.

The chain of implications between the various notions of convergence are noted in their respective sections. They are, using the arrow notation:

: $\begin \xrightarrow & \underset & \xrightarrow & & \\ & & \Downarrow & & \\ \xrightarrow & \Rightarrow & \xrightarrow & \Rightarrow & \xrightarrow \end$

These properties, together with a number of other special cases, are summarized in the following list:

* Almost sure convergence implies convergence in probability: <sup>roof/sup>
*:$X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in probability implies there exists a sub-sequence $(n_k)$ which almost surely converges:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_\ \xrightarrow\ X$
* Convergence in probability implies convergence in distribution: roof/sup>
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in ''r''-th order mean implies convergence in probability:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in ''r''-th order mean implies convergence in lower order mean, assuming that both orders are greater than or equal to one:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X,$ provided ''r'' ≥ ''s'' ≥ 1.
* If ''X''_''n'' converges in distribution to a constant ''c'', then ''X''_''n'' converges in probability to ''c'': roof/sup>
*: $X_n\ \xrightarrow\ c \quad\Rightarrow\quad X_n\ \xrightarrow\ c,$ provided ''c'' is a constant.
* If converges in distribution to ''X'' and the difference between ''X_n'' and ''Y_n'' converges in probability to zero, then ''Y_n'' also converges in distribution to ''X'': roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ , X_n-Y_n, \ \xrightarrow\ 0\ \quad\Rightarrow\quad Y_n\ \xrightarrow\ X$
* If converges in distribution to ''X'' and ''Y_n'' converges in distribution to a constant ''c'', then the joint vector converges in distribution to : roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ c\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,c)$ provided ''c'' is a constant.
*:Note that the condition that converges to a constant is important, if it were to converge to a random variable ''Y'' then we wouldn't be able to conclude that converges to .
* If ''X_n'' converges in probability to ''X'' and ''Y_n'' converges in probability to ''Y'', then the joint vector converges in probability to : roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ Y\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,Y)$
* If converges in probability to ''X'', and if for all ''n'' and some ''b'', then converges in ''r''th mean to ''X'' for all . In other words, if converges in probability to ''X'' and all random variables are almost surely bounded above and below, then converges to ''X'' also in any ''r''th mean.
* Almost sure representation. Usually, convergence in distribution does not imply convergence almost surely. However, for a given sequence which converges in distribution to ''X''₀ it is always possible to find a new probability space (Ω, ''F'', P) and random variables defined on it such that ''Y_n'' is equal in distribution to for each , and ''Y_n'' converges to ''Y''₀ almost surely.
* If for all ''ε'' > 0,
*::$\sum_n \mathbb \left(, X_n - X, > \varepsilon\right) < \infty,$
*:then we say that ''converges almost completely'', or ''almost in probability'' towards ''X''. When converges almost completely towards ''X'' then it also converges almost surely to ''X''. In other words, if converges in probability to ''X'' sufficiently quickly (i.e. the above sequence of tail probabilities is summable for all ), then also converges almost surely to ''X''. This is a direct implication from the Borel–Cantelli lemma.
* If is a sum of ''n'' real independent random variables:
*::$S_n = X_1+\cdots+X_n \,$
*:then converges almost surely if and only if converges in probability. The proof can be found in Page 126 (Theorem 5.3.4) of the book by Kai Lai Chung.
*:However, for a sequence of mutually independent random variables, convergence in probability does not imply almost sure convergence.
* The dominated convergence theorem gives sufficient conditions for almost sure convergence to imply ''L''1-convergence:
}</sup>

*A necessary and sufficient condition for ''L''¹ convergence is $X_n\xrightarrow X$ and the sequence (''X_n'') is uniformly integrable.
*If $X_n\ \xrightarrow\ X$, the followings are equivalent
**$X_n\ \xrightarrow\ X$,
** \mathbb X, ^r
The converse is not necessarily true, however it is true if $\overset$ (by a more general version of Scheffé's lemma).

Properties

Provided the probability space is complete:
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $X_n\ \xrightarrow\ Y$, then $X=Y$ almost surely.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ) and $X_n Y_n\xrightarrow\ XY$.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ) and $X_n Y_n\xrightarrow\ XY$.
* If $X_n\ \xrightarrow\ X$ and $Y_n\ \xrightarrow\ Y$, then $aX_n+bY_n\ \xrightarrow\ aX+bY$ (for any real numbers and ).
* None of the above statements are true for convergence in distribution.

The chain of implications between the various notions of convergence are noted in their respective sections. They are, using the arrow notation:

: $\begin \xrightarrow & \underset & \xrightarrow & & \\ & & \Downarrow & & \\ \xrightarrow & \Rightarrow & \xrightarrow & \Rightarrow & \xrightarrow \end$

These properties, together with a number of other special cases, are summarized in the following list:

* Almost sure convergence implies convergence in probability: <sup>roof/sup>
*:$X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in probability implies there exists a sub-sequence $(n_k)$ which almost surely converges:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_\ \xrightarrow\ X$
* Convergence in probability implies convergence in distribution: roof/sup>
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in ''r''-th order mean implies convergence in probability:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X$
* Convergence in ''r''-th order mean implies convergence in lower order mean, assuming that both orders are greater than or equal to one:
*: $X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X,$ provided ''r'' ≥ ''s'' ≥ 1.
* If ''X''_''n'' converges in distribution to a constant ''c'', then ''X''_''n'' converges in probability to ''c'': roof/sup>
*: $X_n\ \xrightarrow\ c \quad\Rightarrow\quad X_n\ \xrightarrow\ c,$ provided ''c'' is a constant.
* If converges in distribution to ''X'' and the difference between ''X_n'' and ''Y_n'' converges in probability to zero, then ''Y_n'' also converges in distribution to ''X'': roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ , X_n-Y_n, \ \xrightarrow\ 0\ \quad\Rightarrow\quad Y_n\ \xrightarrow\ X$
* If converges in distribution to ''X'' and ''Y_n'' converges in distribution to a constant ''c'', then the joint vector converges in distribution to : roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ c\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,c)$ provided ''c'' is a constant.
*:Note that the condition that converges to a constant is important, if it were to converge to a random variable ''Y'' then we wouldn't be able to conclude that converges to .
* If ''X_n'' converges in probability to ''X'' and ''Y_n'' converges in probability to ''Y'', then the joint vector converges in probability to : roof/sup>
*: $X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ Y\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,Y)$
* If converges in probability to ''X'', and if for all ''n'' and some ''b'', then converges in ''r''th mean to ''X'' for all . In other words, if converges in probability to ''X'' and all random variables are almost surely bounded above and below, then converges to ''X'' also in any ''r''th mean.
* Almost sure representation. Usually, convergence in distribution does not imply convergence almost surely. However, for a given sequence which converges in distribution to ''X''₀ it is always possible to find a new probability space (Ω, ''F'', P) and random variables defined on it such that ''Y_n'' is equal in distribution to for each , and ''Y_n'' converges to ''Y''₀ almost surely.
* If for all ''ε'' > 0,
*::$\sum_n \mathbb \left(, X_n - X, > \varepsilon\right) < \infty,$
*:then we say that ''converges almost completely'', or ''almost in probability'' towards ''X''. When converges almost completely towards ''X'' then it also converges almost surely to ''X''. In other words, if converges in probability to ''X'' sufficiently quickly (i.e. the above sequence of tail probabilities is summable for all ), then also converges almost surely to ''X''. This is a direct implication from the Borel–Cantelli lemma.
* If is a sum of ''n'' real independent random variables:
*::$S_n = X_1+\cdots+X_n \,$
*:then converges almost surely if and only if converges in probability. The proof can be found in Page 126 (Theorem 5.3.4) of the book by Kai Lai Chung.
*:However, for a sequence of mutually independent random variables, convergence in probability does not imply almost sure convergence.
* The dominated convergence theorem gives sufficient conditions for almost sure convergence to imply ''L''1-convergence:
}</sup>

*A necessary and sufficient condition for ''L''¹ convergence is $X_n\xrightarrow X$ and the sequence (''X_n'') is uniformly integrable.
*If $X_n\ \xrightarrow\ X$, the followings are equivalent
**$X_n\ \xrightarrow\ X$,
** \mathbb X_n, ^r\rightarrow \mathbb X, ^r< \infty ,
**$\$ is uniformly integrable.

See also

* Proofs of convergence of random variables
* Convergence of measures
* Convergence in measure
* Continuous stochastic process: the question of continuity of a stochastic process is essentially a question of convergence, and many of the same concepts and relationships used above apply to the continuity question.
* Asymptotic distribution
* Big O in probability notation
* Skorokhod's representation theorem
* The Tweedie convergence theorem
* Slutsky's theorem
* Continuous mapping theorem

Notes

References

*
*
*
*
*
*
*
*
*
*
*
*
*
*
*

{{DEFAULTSORT:Convergence Of Random Variables
Stochastic processes
Random variables, Convergence of>X_n, ^r\rightarrow \mathbb X, ^r< \infty ,
**$\$ is uniformly integrable.

See also

* Proofs of convergence of random variables
* Convergence of measures
* Convergence in measure
* Continuous stochastic process: the question of continuity of a stochastic process is essentially a question of convergence, and many of the same concepts and relationships used above apply to the continuity question.
* Asymptotic distribution
* Big O in probability notation
* Skorokhod's representation theorem
* The Tweedie convergence theorem
* Slutsky's theorem
* Continuous mapping theorem

Notes

References

*
*
*
*
*
*
*
*
*
*
*
*
*
*
*

{{DEFAULTSORT:Convergence Of Random Variables
Stochastic processes
Random variables, Convergence of>X_n, ^r\rightarrow \mathbb X, ^r< \infty ,
**$\$ is uniformly integrable.

See also

* Proofs of convergence of random variables
* Convergence of measures
* Convergence in measure
* Continuous stochastic process: the question of continuity of a stochastic process is essentially a question of convergence, and many of the same concepts and relationships used above apply to the continuity question.
* Asymptotic distribution
* Big O in probability notation
* Skorokhod's representation theorem
* The Tweedie convergence theorem
* Slutsky's theorem
* Continuous mapping theorem

Notes

References

*
*
*
*
*
*
*
*
*
*
*
*
*
*
*

{{DEFAULTSORT:Convergence Of Random Variables
Stochastic processes
Random variables, Convergence of>X_n, ^r\rightarrow \mathbb X, ^r< \infty ,
**$\$ is uniformly integrable.

See also

* Proofs of convergence of random variables
* Convergence of measures
* Convergence in measure
* Continuous stochastic process: the question of continuity of a stochastic process is essentially a question of convergence, and many of the same concepts and relationships used above apply to the continuity question.
* Asymptotic distribution
* Big O in probability notation
* Skorokhod's representation theorem
* The Tweedie convergence theorem
* Slutsky's theorem
* Continuous mapping theorem

Notes

References

*
*
*
*
*
*
*
*
*
*
*
*
*
*
*

{{DEFAULTSORT:Convergence Of Random Variables
Stochastic processes
Convergence (mathematics), Random variables, Convergence of