HOME

TheInfoList



OR:

In
probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set ...
, there exist several different notions of convergence of random variables. The
convergence Convergence may refer to: Arts and media Literature *''Convergence'' (book series), edited by Ruth Nanda Anshen *Convergence (comics), "Convergence" (comics), two separate story lines published by DC Comics: **A four-part crossover storyline that ...
of
sequence In mathematics, a sequence is an enumerated collection of objects in which repetitions are allowed and order matters. Like a set, it contains members (also called ''elements'', or ''terms''). The number of elements (possibly infinite) is called ...
s of
random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...
s to some
limit Limit or Limits may refer to: Arts and media * ''Limit'' (manga), a manga by Keiko Suenobu * ''Limit'' (film), a South Korean film * Limit (music), a way to characterize harmony * "Limit" (song), a 2016 single by Luna Sea * "Limits", a 2019 ...
random variable is an important concept in probability theory, and its applications to
statistics Statistics (from German: '' Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, indust ...
and
stochastic process In probability theory and related fields, a stochastic () or random process is a mathematical object usually defined as a family of random variables. Stochastic processes are widely used as mathematical models of systems and phenomena that ap ...
es. The same concepts are known in more general
mathematics Mathematics is an area of knowledge that includes the topics of numbers, formulas and related structures, shapes and the spaces in which they are contained, and quantities and their changes. These topics are represented in modern mathematics ...
as stochastic convergence and they formalize the idea that a sequence of essentially random or unpredictable events can sometimes be expected to settle down into a behavior that is essentially unchanging when items far enough into the sequence are studied. The different possible notions of convergence relate to how such a behavior can be characterized: two readily understood behaviors are that the sequence eventually takes a constant value, and that values in the sequence continue to change but can be described by an unchanging probability distribution.


Background

"Stochastic convergence" formalizes the idea that a sequence of essentially random or unpredictable events can sometimes be expected to settle into a pattern. The pattern may for instance be *
Convergence Convergence may refer to: Arts and media Literature *''Convergence'' (book series), edited by Ruth Nanda Anshen *Convergence (comics), "Convergence" (comics), two separate story lines published by DC Comics: **A four-part crossover storyline that ...
in the classical sense to a fixed value, perhaps itself coming from a random event *An increasing similarity of outcomes to what a purely deterministic function would produce *An increasing preference towards a certain outcome *An increasing "aversion" against straying far away from a certain outcome *That the probability distribution describing the next outcome may grow increasingly similar to a certain distribution Some less obvious, more theoretical patterns could be *That the series formed by calculating the
expected value In probability theory, the expected value (also called expectation, expectancy, mathematical expectation, mean, average, or first moment) is a generalization of the weighted average. Informally, the expected value is the arithmetic mean of a ...
of the outcome's distance from a particular value may converge to 0 *That the variance of the
random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...
describing the next event grows smaller and smaller. These other types of patterns that may arise are reflected in the different types of stochastic convergence that have been studied. While the above discussion has related to the convergence of a single series to a limiting value, the notion of the convergence of two series towards each other is also important, but this is easily handled by studying the sequence defined as either the difference or the ratio of the two series. For example, if the average of ''n''
independent Independent or Independents may refer to: Arts, entertainment, and media Artist groups * Independents (artist group), a group of modernist painters based in the New Hope, Pennsylvania, area of the United States during the early 1930s * Independe ...
random variables ''Y''''i'', ''i'' = 1, ..., ''n'', all having the same finite
mean There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value ( magnitude and sign) of a given data set. For a data set, the '' ar ...
and
variance In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbe ...
, is given by :X_n = \frac\sum_^n Y_i\,, then as ''n'' tends to infinity, converges ''in probability'' (see below) to the common
mean There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value ( magnitude and sign) of a given data set. For a data set, the '' ar ...
, μ, of the random variables ''Y''''i''. This result is known as the
weak law of large numbers In probability theory, the law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials shou ...
. Other forms of convergence are important in other useful theorems, including the
central limit theorem In probability theory, the central limit theorem (CLT) establishes that, in many situations, when independent random variables are summed up, their properly normalized sum tends toward a normal distribution even if the original variables themsel ...
. Throughout the following, we assume that (''X''''n'') is a sequence of random variables, and ''X'' is a random variable, and all of them are defined on the same
probability space In probability theory, a probability space or a probability triple (\Omega, \mathcal, P) is a mathematical construct that provides a formal model of a random process or "experiment". For example, one can define a probability space which models t ...
(\Omega, \mathcal, \operatorname ).


Convergence in distribution

With this mode of convergence, we increasingly expect to see the next outcome in a sequence of random experiments becoming better and better modeled by a given
probability distribution In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon ...
. Convergence in distribution is the weakest form of convergence typically discussed, since it is implied by all other types of convergence mentioned in this article. However, convergence in distribution is very frequently used in practice; most often it arises from application of the
central limit theorem In probability theory, the central limit theorem (CLT) establishes that, in many situations, when independent random variables are summed up, their properly normalized sum tends toward a normal distribution even if the original variables themsel ...
.


Definition

A sequence X_1, X_2, \ldots of real-valued
random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...
s, with
cumulative distribution function In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x. Eve ...
s F_1, F_2, \ldots , is said to converge in distribution, or converge weakly, or converge in law to a random variable with
cumulative distribution function In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x. Eve ...
if : \lim_ F_n(x) = F(x), for every number x \in \mathbb at which is
continuous Continuity or continuous may refer to: Mathematics * Continuity (mathematics), the opposing concept to discreteness; common examples include ** Continuous probability distribution or random variable in probability and statistics ** Continuous g ...
. The requirement that only the continuity points of should be considered is essential. For example, if are distributed uniformly on intervals , then this sequence converges in distribution to the degenerate random variable . Indeed, for all ''n'' when , and for all when . However, for this limiting random variable , even though for all . Thus the convergence of cdfs fails at the point where is discontinuous. Convergence in distribution may be denoted as where \scriptstyle\mathcal_X is the law (probability distribution) of . For example, if is standard normal we can write X_n\,\xrightarrow\,\mathcal(0,\,1). For
random vector In probability, and statistics, a multivariate random variable or random vector is a list of mathematical variables each of whose value is unknown, either because the value has not yet occurred or because there is imperfect knowledge of its value ...
s the convergence in distribution is defined similarly. We say that this sequence converges in distribution to a random -vector if : \lim_ \operatorname(X_n\in A) = \operatorname(X\in A) for every which is a
continuity set In measure theory, a branch of mathematics, a continuity set of a measure ''μ'' is any Borel set ''B'' such that : \mu(\partial B) = 0\,, where \partial B is the (topological) boundary of ''B''. For signed measures, one asks that : , \mu, (\ ...
of . The definition of convergence in distribution may be extended from random vectors to more general
random element In probability theory, random element is a generalization of the concept of random variable to more complicated spaces than the simple real line. The concept was introduced by who commented that the “development of probability theory and expansi ...
s in arbitrary
metric space In mathematics, a metric space is a set together with a notion of '' distance'' between its elements, usually called points. The distance is measured by a function called a metric or distance function. Metric spaces are the most general setti ...
s, and even to the “random variables” which are not measurable — a situation which occurs for example in the study of
empirical process In probability theory, an empirical process is a stochastic process that describes the proportion of objects in a system in a given state. For a process in a discrete state space a population continuous time Markov chain or Markov population model ...
es. This is the “weak convergence of laws without laws being defined” — except asymptotically. In this case the term weak convergence is preferable (see weak convergence of measures), and we say that a sequence of random elements converges weakly to (denoted as ) if : \operatorname^*h(X_n) \to \operatorname\,h(X) for all continuous bounded functions . Here E* denotes the ''outer expectation'', that is the expectation of a “smallest measurable function that dominates ”.


Properties

* Since , the convergence in distribution means that the probability for to be in a given range is approximately equal to the probability that the value of is in that range, provided is sufficiently large. *In general, convergence in distribution does not imply that the sequence of corresponding
probability density function In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) ca ...
s will also converge. As an example one may consider random variables with densities . These random variables converge in distribution to a uniform ''U''(0, 1), whereas their densities do not converge at all. ** However, according to ''Scheffé’s theorem'', convergence of the
probability density function In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) ca ...
s implies convergence in distribution. * The portmanteau lemma provides several equivalent definitions of convergence in distribution. Although these definitions are less intuitive, they are used to prove a number of statistical theorems. The lemma states that converges in distribution to if and only if any of the following statements are true: ** \Pr(X_n \le x) \to \Pr(X \le x) for all continuity points of x\mapsto \Pr(X \le x); ** \operatornamef(X_n) \to \operatornamef(X) for all bounded,
continuous function In mathematics, a continuous function is a function such that a continuous variation (that is a change without jump) of the argument induces a continuous variation of the value of the function. This means that there are no abrupt changes in val ...
s f (where \operatorname denotes the
expected value In probability theory, the expected value (also called expectation, expectancy, mathematical expectation, mean, average, or first moment) is a generalization of the weighted average. Informally, the expected value is the arithmetic mean of a ...
operator); ** \operatornamef(X_n) \to \operatornamef(X) for all bounded,
Lipschitz function In mathematical analysis, Lipschitz continuity, named after German mathematician Rudolf Lipschitz, is a strong form of uniform continuity for functions. Intuitively, a Lipschitz continuous function is limited in how fast it can change: there e ...
s f; ** \lim\inf \operatornamef(X_n) \ge \operatornamef(X) for all nonnegative, continuous functions f; ** \lim\inf \Pr(X_n \in G) \ge \Pr(X \in G) for every
open set In mathematics, open sets are a generalization of open intervals in the real line. In a metric space (a set along with a distance defined between any two points), open sets are the sets that, with every point , contain all points that are su ...
G; ** \lim\sup \Pr(X_n \in F) \le \Pr(X \in F) for every
closed set In geometry, topology, and related branches of mathematics, a closed set is a set whose complement is an open set. In a topological space, a closed set can be defined as a set which contains all its limit points. In a complete metric space, a ...
F; ** \Pr(X_n \in B) \to \Pr(X \in B) for all
continuity set In measure theory, a branch of mathematics, a continuity set of a measure ''μ'' is any Borel set ''B'' such that : \mu(\partial B) = 0\,, where \partial B is the (topological) boundary of ''B''. For signed measures, one asks that : , \mu, (\ ...
s B of random variable X; ** \limsup \operatornamef(X_n) \le \operatornamef(X) for every upper semi-continuous function f bounded above; ** \liminf \operatornamef(X_n) \ge \operatornamef(X) for every
lower semi-continuous In mathematical analysis, semicontinuity (or semi-continuity) is a property of extended real-valued functions that is weaker than continuity. An extended real-valued function f is upper (respectively, lower) semicontinuous at a point x_0 if, ro ...
function f bounded below. * The continuous mapping theorem states that for a continuous function , if the sequence converges in distribution to , then converges in distribution to . ** Note however that convergence in distribution of to and to does in general ''not'' imply convergence in distribution of to or of to . * Lévy’s continuity theorem: the sequence converges in distribution to if and only if the sequence of corresponding
characteristic function In mathematics, the term "characteristic function" can refer to any of several distinct concepts: * The indicator function of a subset, that is the function ::\mathbf_A\colon X \to \, :which for a given subset ''A'' of ''X'', has value 1 at points ...
s converges pointwise to the characteristic function of . * Convergence in distribution is
metrizable In topology and related areas of mathematics, a metrizable space is a topological space that is homeomorphic to a metric space. That is, a topological space (X, \mathcal) is said to be metrizable if there is a metric d : X \times X \to , \infty) s ...
by the Lévy–Prokhorov metric. * A natural link to convergence in distribution is the
Skorokhod's representation theorem In mathematics and statistics, Skorokhod's representation theorem is a result that shows that a weakly convergent sequence of probability measures whose limit measure is sufficiently well-behaved can be represented as the distribution/law of a po ...
.


Convergence in probability

The basic idea behind this type of convergence is that the probability of an “unusual” outcome becomes smaller and smaller as the sequence progresses. The concept of convergence in probability is used very often in statistics. For example, an estimator is called consistent if it converges in probability to the quantity being estimated. Convergence in probability is also the type of convergence established by the
weak law of large numbers In probability theory, the law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials shou ...
.


Definition

A sequence of random variables converges in probability towards the random variable ''X'' if for all ''ε'' > 0 : \lim_\Pr\big(, X_n-X, > \varepsilon\big) = 0. More explicitly, let ''P''''n''(''ε'') be the probability that ''X''''n'' is outside the ball of radius ''ε'' centered at ''X''. Then is said to converge in probability to ''X'' if for any and any ''δ'' > 0 there exists a number ''N'' (which may depend on ''ε'' and ''δ'') such that for all ''n'' ≥ ''N'', ''P''''n''(''ε'') < ''δ'' (the definition of limit). Notice that for the condition to be satisfied, it is not possible that for each ''n'' the random variables ''X'' and ''X''''n'' are independent (and thus convergence in probability is a condition on the joint cdf's, as opposed to convergence in distribution, which is a condition on the individual cdf's), unless ''X'' is deterministic like for the weak law of large numbers. At the same time, the case of a deterministic ''X'' cannot, whenever the deterministic value is a discontinuity point (not isolated), be handled by convergence in distribution, where discontinuity points have to be explicitly excluded. Convergence in probability is denoted by adding the letter ''p'' over an arrow indicating convergence, or using the "plim" probability limit operator: For random elements on a
separable metric space In mathematics, a topological space is called separable if it contains a countable, dense subset; that is, there exists a sequence \_^ of elements of the space such that every nonempty open subset of the space contains at least one element of th ...
, convergence in probability is defined similarly by : \forall\varepsilon>0, \Pr\big(d(X_n,X)\geq\varepsilon\big) \to 0.


Properties

* Convergence in probability implies convergence in distribution. roof/sup> * In the opposite direction, convergence in distribution implies convergence in probability when the limiting random variable ''X'' is a constant. roof/sup> * Convergence in probability does not imply almost sure convergence. roof/sup> * The continuous mapping theorem states that for every continuous function g, if X_n \xrightarrow X, then also  * Convergence in probability defines a
topology In mathematics, topology (from the Greek words , and ) is concerned with the properties of a geometric object that are preserved under continuous deformations, such as stretching, twisting, crumpling, and bending; that is, without closing ...
on the space of random variables over a fixed probability space. This topology is
metrizable In topology and related areas of mathematics, a metrizable space is a topological space that is homeomorphic to a metric space. That is, a topological space (X, \mathcal) is said to be metrizable if there is a metric d : X \times X \to , \infty) s ...
by the ''
Ky Fan Ky Fan (樊𰋀, , September 19, 1914 – March 22, 2010) was a Chinese-born American mathematician. He was a professor of mathematics at the University of California, Santa Barbara. Biography Fan was born in Hangzhou, the capital of Zhejiang ...
metric'': d(X,Y) = \inf\!\big\ or alternately by this metric d(X,Y)=\mathbb E\left X-Y, , 1)\right


Almost sure convergence

This is the type of stochastic convergence that is most similar to
pointwise convergence In mathematics, pointwise convergence is one of various senses in which a sequence of functions can converge to a particular function. It is weaker than uniform convergence, to which it is often compared. Definition Suppose that X is a set an ...
known from elementary
real analysis In mathematics, the branch of real analysis studies the behavior of real numbers, sequences and series of real numbers, and real functions. Some particular properties of real-valued sequences and functions that real analysis studies include conv ...
.


Definition

To say that the sequence converges almost surely or almost everywhere or with probability 1 or strongly towards ''X'' means that : \operatorname\!\left( \lim_\! X_n = X \right) = 1. This means that the values of approach the value of ''X'', in the sense (see
almost surely In probability theory, an event is said to happen almost surely (sometimes abbreviated as a.s.) if it happens with probability 1 (or Lebesgue measure 1). In other words, the set of possible exceptions may be non-empty, but it has probability 0. ...
) that events for which does not converge to ''X'' have probability 0. Using the probability space (\Omega, \mathcal, \operatorname ) and the concept of the random variable as a function from Ω to R, this is equivalent to the statement : \operatorname\Big( \omega \in \Omega : \lim_ X_n(\omega) = X(\omega) \Big) = 1. Using the notion of the limit superior of a sequence of sets, almost sure convergence can also be defined as follows: : \operatorname\Big( \limsup_ \big\ \Big) = 0 \quad\text\quad \varepsilon>0. Almost sure convergence is often denoted by adding the letters ''a.s.'' over an arrow indicating convergence: For generic
random element In probability theory, random element is a generalization of the concept of random variable to more complicated spaces than the simple real line. The concept was introduced by who commented that the “development of probability theory and expansi ...
s on a
metric space In mathematics, a metric space is a set together with a notion of '' distance'' between its elements, usually called points. The distance is measured by a function called a metric or distance function. Metric spaces are the most general setti ...
(S,d), convergence almost surely is defined similarly: : \operatorname\Big( \omega\in\Omega:\, d\big(X_n(\omega),X(\omega)\big)\,\underset\,0 \Big) = 1


Properties

* Almost sure convergence implies convergence in probability (by
Fatou's lemma In mathematics, Fatou's lemma establishes an inequality relating the Lebesgue integral of the limit inferior of a sequence of functions to the limit inferior of integrals of these functions. The lemma is named after Pierre Fatou. Fatou's le ...
), and hence implies convergence in distribution. It is the notion of convergence used in the strong
law of large numbers In probability theory, the law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials shou ...
. * The concept of almost sure convergence does not come from a
topology In mathematics, topology (from the Greek words , and ) is concerned with the properties of a geometric object that are preserved under continuous deformations, such as stretching, twisting, crumpling, and bending; that is, without closing ...
on the space of random variables. This means there is no topology on the space of random variables such that the almost surely convergent sequences are exactly the converging sequences with respect to that topology. In particular, there is no metric of almost sure convergence.


Sure convergence or pointwise convergence

To say that the sequence of
random variables A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...
(''X''''n'') defined over the same
probability space In probability theory, a probability space or a probability triple (\Omega, \mathcal, P) is a mathematical construct that provides a formal model of a random process or "experiment". For example, one can define a probability space which models t ...
(i.e., a
random process In probability theory and related fields, a stochastic () or random process is a mathematical object usually defined as a family of random variables. Stochastic processes are widely used as mathematical models of systems and phenomena that appea ...
) converges surely or everywhere or pointwise towards ''X'' means \lim_ X_n(\omega) = X(\omega), \, \, \forall \omega \in \Omega. where Ω is the
sample space In probability theory, the sample space (also called sample description space, possibility space, or outcome space) of an experiment or random trial is the set of all possible outcomes or results of that experiment. A sample space is usually den ...
of the underlying
probability space In probability theory, a probability space or a probability triple (\Omega, \mathcal, P) is a mathematical construct that provides a formal model of a random process or "experiment". For example, one can define a probability space which models t ...
over which the random variables are defined. This is the notion of
pointwise convergence In mathematics, pointwise convergence is one of various senses in which a sequence of functions can converge to a particular function. It is weaker than uniform convergence, to which it is often compared. Definition Suppose that X is a set an ...
of a sequence of functions extended to a sequence of
random variables A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...
. (Note that random variables themselves are functions). \left\ = \Omega. Sure convergence of a random variable implies all the other kinds of convergence stated above, but there is no payoff in
probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set ...
by using sure convergence compared to using almost sure convergence. The difference between the two only exists on sets with probability zero. This is why the concept of sure convergence of random variables is very rarely used.


Convergence in mean

Given a real number , we say that the sequence converges in the ''r''-th mean (or in the ''Lr''-norm) towards the random variable ''X'', if the -th absolute moments E(, ''Xn'', ''r '') and E(, ''X'', ''r '') of and ''X'' exist, and : \lim_ \operatorname\left( , X_n-X, ^r \right) = 0, where the operator E denotes the
expected value In probability theory, the expected value (also called expectation, expectancy, mathematical expectation, mean, average, or first moment) is a generalization of the weighted average. Informally, the expected value is the arithmetic mean of a ...
. Convergence in -th mean tells us that the expectation of the -th power of the difference between X_n and X converges to zero. This type of convergence is often denoted by adding the letter ''Lr'' over an arrow indicating convergence: The most important cases of convergence in ''r''-th mean are: * When converges in ''r''-th mean to ''X'' for ''r'' = 1, we say that converges in mean to ''X''. * When converges in ''r''-th mean to ''X'' for ''r'' = 2, we say that converges in mean square (or in quadratic mean) to ''X''. Convergence in the ''r''-th mean, for ''r'' ≥ 1, implies convergence in probability (by Markov's inequality). Furthermore, if ''r'' > ''s'' ≥ 1, convergence in ''r''-th mean implies convergence in ''s''-th mean. Hence, convergence in mean square implies convergence in mean. It is also worth noticing that if then : \lim_ E X, ^r


_Properties

Provided_the_probability_space_is_
complete Complete may refer to: Logic * Completeness (logic) * Completeness of a theory, the property of a theory that every formula in the theory's language or its negation is provable Mathematics * The completeness of the real numbers, which implies t ...
: *_If_X_n\_\xrightarrow\_X_and_X_n\_\xrightarrow\_Y,_then_X=Y_almost_surely_ In_probability_theory,_an_event_is_said_to_happen_almost_surely_(sometimes_abbreviated_as_a.s.)_if_it_happens_with_probability_1_(or__Lebesgue_measure_1)._In_other_words,_the_set_of_possible_exceptions_may_be_non-empty,_but_it_has_probability_0._...
. *_If_X_n\_\xrightarrow\_X_and_X_n\_\xrightarrow\_Y,_then_X=Y_almost_surely. *_If_X_n\_\xrightarrow\_X_and_X_n\_\xrightarrow\_Y,_then_X=Y_almost_surely. *_If_X_n\_\xrightarrow\_X_and_Y_n\_\xrightarrow\_Y,_then_aX_n+bY_n\_\xrightarrow\_aX+bY_(for_any_real_numbers__and_)_and_X_n_Y_n\xrightarrow\_XY. *_If_X_n\_\xrightarrow\_X_and_Y_n\_\xrightarrow\_Y,_then_aX_n+bY_n\_\xrightarrow\_aX+bY_(for_any_real_numbers__and_)_and_X_n_Y_n\xrightarrow\_XY. *_If_X_n\_\xrightarrow\_X_and_Y_n\_\xrightarrow\_Y,_then_aX_n+bY_n\_\xrightarrow\_aX+bY_(for_any_real_numbers__and_). *_None_of_the_above_statements_are_true_for_convergence_in_distribution. _ The_chain_of_implications_between_the_various_notions_of_convergence_are_noted_in_their_respective_sections._They_are,_using_the_arrow_notation: :_\begin __\xrightarrow__&_\underset_&__\xrightarrow__&_____________&_\\ _________________________________&__________________________________&_____\Downarrow__________________&_____________&_\\ __\xrightarrow______&____________\Rightarrow___________&__\xrightarrow________________&_\Rightarrow_&_\xrightarrow __\end These_properties,_together_with_a_number_of_other_special_cases,_are_summarized_in_the_following_list: *__Almost_sure_convergence_implies_convergence_in_probability: roof/sup> *:X_n\_\xrightarrow\_X__\quad\Rightarrow\quad__X_n\_\xrightarrow\_X *_Convergence_in_probability_implies_there_exists_a_sub-sequence_(n_k)_which_almost_surely_converges:_ *:_X_n\_\xrightarrow\_X__\quad\Rightarrow\quad__X_\_\xrightarrow\_X *_Convergence_in_probability_implies_convergence_in_distribution:__.html"_;"title="roof">roof/sup> *:_X_n\_\xrightarrow\_X_\quad\Rightarrow\quad__X_n\_\xrightarrow\_X *___Convergence_in_''r''-th_order_mean_implies_convergence_in_probability: *:_X_n\_\xrightarrow\_X__\quad\Rightarrow\quad__X_n\_\xrightarrow\_X *___Convergence_in_''r''-th_order_mean_implies_convergence_in_lower_order_mean,_assuming_that_both_orders_are_greater_than_or_equal_to_one: *:_X_n\_\xrightarrow\_X__\quad\Rightarrow\quad__X_n\_\xrightarrow\_X,_provided_''r''_≥_''s''_≥_1. *___If_''X''''n''_converges_in_distribution_to_a_constant_''c'',_then_''X''''n''_converges_in_probability_to_''c'':__.html"_;"title="roof">roof/sup> *:_X_n\_\xrightarrow\_c_\quad\Rightarrow\quad_X_n\_\xrightarrow\_c,_provided_''c''_is_a_constant. *___If__converges_in_distribution_to_''X''_and_the_difference_between_''Xn''_and_''Yn''_converges_in_probability_to_zero,_then_''Yn''_also_converges_in_distribution_to_''X'': roof/sup> *:_X_n\_\xrightarrow\_X,\_\_, X_n-Y_n, \_\xrightarrow\_0\__\quad\Rightarrow\quad__Y_n\_\xrightarrow\_X *___If__converges_in_distribution_to_''X''_and_''Yn''_converges_in_distribution_to_a_constant_''c'',_then_the_joint_vector__converges_in_distribution_to_: roof/sup> *:_X_n\_\xrightarrow\_X,\_\_Y_n\_\xrightarrow\_c\_\quad\Rightarrow\quad_(X_n,Y_n)\_\xrightarrow\_(X,c)_provided_''c''_is_a_constant. *:Note_that_the_condition_that__converges_to_a_constant_is_important,_if_it_were_to_converge_to_a_random_variable_''Y''_then_we_wouldn't_be_able_to_conclude_that__converges_to_. *___If_''Xn''_converges_in_probability_to_''X''_and_''Yn''_converges_in_probability_to_''Y'',_then_the_joint_vector__converges_in_probability_to_: roof/sup> *:_X_n\_\xrightarrow\_X,\_\_Y_n\_\xrightarrow\_Y\_\quad\Rightarrow\quad_(X_n,Y_n)\_\xrightarrow\_(X,Y) *_If__converges_in_probability_to_''X'',_and_if__for_all_''n''_and_some_''b'',_then__converges_in_''r''th_mean_to_''X''_for_all_._In_other_words,_if__converges_in_probability_to_''X''_and_all_random_variables__are_almost_surely_bounded_above_and_below,_then__converges_to_''X''_also_in_any_''r''th_mean. *_Almost_sure_representation._Usually,_convergence_in_distribution_does_not_imply_convergence_almost_surely._However,_for_a_given_sequence__which_converges_in_distribution_to_''X''0_it_is_always_possible_to_find_a_new_probability_space_(Ω,_''F'',_P)_and_random_variables__defined_on_it_such_that_''Yn''_is_equal_in_distribution_to__for_each_,_and_''Yn''_converges_to_''Y''0_almost_surely. *_If_for_all_''ε''_>_0, *::\sum_n_\mathbb_\left(, X_n_-_X, _>_\varepsilon\right)_<_\infty, *:then_we_say_that__''converges_almost_completely'',_or_''almost_in_probability''_towards_''X''._When__converges_almost_completely_towards_''X''_then_it_also_converges_almost_surely_to_''X''.__In_other_words,_if__converges_in_probability_to_''X''_sufficiently_quickly_(i.e._the_above_sequence_of_tail_probabilities_is_summable_for_all_),_then__also_converges_almost_surely_to_''X''._This_is_a_direct_implication_from_the_ Borel–Cantelli_lemma. *_If__is_a_sum_of_''n''_real_independent_random_variables: *::S_n_=_X_1+\cdots+X_n_\,_ *:then__converges_almost_surely_if_and_only_if__converges_in_probability. *_The_ dominated_convergence_theorem_gives_sufficient_conditions_for_almost_sure_convergence_to_imply_''L''1-convergence: } *A_necessary_and_sufficient_condition_for__''L''1_convergence_is_X_n\xrightarrow_X_and_the_sequence_(''Xn'')_is_ uniformly_integrable. *If_X_n_are_discrete_and_independent,_then_X_n_\stackrel_X_implies_that_X_n_\stackrel_X._This_is_a_consequence_of_the_ second_Borel–Cantelli_lemma.


_See_also

*_ Proofs_of_convergence_of_random_variables *_ Convergence_of_measures *_ Convergence_in_measure *_ Continuous_stochastic_process:_the_question_of_continuity_of_a_stochastic_process_ In_probability_theory_and_related_fields,_a_stochastic_()_or_random_process_is_a__mathematical_object_usually_defined_as_a__family_of__random_variables._Stochastic_processes_are_widely_used_as_mathematical_models_of_systems_and_phenomena_that_ap_...
_is_essentially_a_question_of_convergence,_and_many_of_the_same_concepts_and_relationships_used_above_apply_to_the_continuity_question. *_
Asymptotic_distribution In mathematics and statistics, an asymptotic distribution is a probability distribution that is in a sense the "limiting" distribution of a sequence of distributions. One of the main uses of the idea of an asymptotic distribution is in providing ...
*_
Big_O_in_probability_notation The order in probability notation is used in probability theory and statistical theory in direct parallel to the big-O notation that is standard in mathematics. Where the big-O notation deals with the convergence of sequences or sets of ordinary n ...
*_Skorokhod's_representation_theorem_In__mathematics_and_statistics,_Skorokhod's_representation_theorem_is_a_result_that_shows_that_a__weakly_convergent_sequence_of_probability_measures_whose_limit_measure_is_sufficiently_well-behaved_can_be_represented_as_the_distribution/law_of_a__po_...
*_ The_Tweedie_convergence_theorem *_
Slutsky's_theorem In probability theory, Slutsky’s theorem extends some properties of algebraic operations on convergent sequences of real numbers to sequences of random variables. The theorem was named after Eugen Slutsky. Slutsky's theorem is also attributed t ...
*_ Continuous_mapping_theorem


__Notes_


__References_

*_ *_ *_ *_ *_ *_ *_ *_ *_ *_ *_ *_ *_ *_ *_https://www.ma.utexas.edu/users/gordanz/notes/weak.pdf {{DEFAULTSORT:Convergence_Of_Random_Variables Stochastic_processes Random_variables,_Convergence_ofhtml" ;"title="X_n, ^r] = E X, ^r


Properties

Provided the probability space is
complete Complete may refer to: Logic * Completeness (logic) * Completeness of a theory, the property of a theory that every formula in the theory's language or its negation is provable Mathematics * The completeness of the real numbers, which implies t ...
: * If X_n\ \xrightarrow\ X and X_n\ \xrightarrow\ Y, then X=Y
almost surely In probability theory, an event is said to happen almost surely (sometimes abbreviated as a.s.) if it happens with probability 1 (or Lebesgue measure 1). In other words, the set of possible exceptions may be non-empty, but it has probability 0. ...
. * If X_n\ \xrightarrow\ X and X_n\ \xrightarrow\ Y, then X=Y almost surely. * If X_n\ \xrightarrow\ X and X_n\ \xrightarrow\ Y, then X=Y almost surely. * If X_n\ \xrightarrow\ X and Y_n\ \xrightarrow\ Y, then aX_n+bY_n\ \xrightarrow\ aX+bY (for any real numbers and ) and X_n Y_n\xrightarrow\ XY. * If X_n\ \xrightarrow\ X and Y_n\ \xrightarrow\ Y, then aX_n+bY_n\ \xrightarrow\ aX+bY (for any real numbers and ) and X_n Y_n\xrightarrow\ XY. * If X_n\ \xrightarrow\ X and Y_n\ \xrightarrow\ Y, then aX_n+bY_n\ \xrightarrow\ aX+bY (for any real numbers and ). * None of the above statements are true for convergence in distribution. The chain of implications between the various notions of convergence are noted in their respective sections. They are, using the arrow notation: : \begin \xrightarrow & \underset & \xrightarrow & & \\ & & \Downarrow & & \\ \xrightarrow & \Rightarrow & \xrightarrow & \Rightarrow & \xrightarrow \end These properties, together with a number of other special cases, are summarized in the following list: * Almost sure convergence implies convergence in probability: roof/sup> *:X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X * Convergence in probability implies there exists a sub-sequence (n_k) which almost surely converges: *: X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_\ \xrightarrow\ X * Convergence in probability implies convergence in distribution: roof/sup> *: X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X * Convergence in ''r''-th order mean implies convergence in probability: *: X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X * Convergence in ''r''-th order mean implies convergence in lower order mean, assuming that both orders are greater than or equal to one: *: X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X, provided ''r'' ≥ ''s'' ≥ 1. * If ''X''''n'' converges in distribution to a constant ''c'', then ''X''''n'' converges in probability to ''c'': roof/sup> *: X_n\ \xrightarrow\ c \quad\Rightarrow\quad X_n\ \xrightarrow\ c, provided ''c'' is a constant. * If converges in distribution to ''X'' and the difference between ''Xn'' and ''Yn'' converges in probability to zero, then ''Yn'' also converges in distribution to ''X'': roof/sup> *: X_n\ \xrightarrow\ X,\ \ , X_n-Y_n, \ \xrightarrow\ 0\ \quad\Rightarrow\quad Y_n\ \xrightarrow\ X * If converges in distribution to ''X'' and ''Yn'' converges in distribution to a constant ''c'', then the joint vector converges in distribution to : roof/sup> *: X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ c\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,c) provided ''c'' is a constant. *:Note that the condition that converges to a constant is important, if it were to converge to a random variable ''Y'' then we wouldn't be able to conclude that converges to . * If ''Xn'' converges in probability to ''X'' and ''Yn'' converges in probability to ''Y'', then the joint vector converges in probability to : roof/sup> *: X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ Y\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,Y) * If converges in probability to ''X'', and if for all ''n'' and some ''b'', then converges in ''r''th mean to ''X'' for all . In other words, if converges in probability to ''X'' and all random variables are almost surely bounded above and below, then converges to ''X'' also in any ''r''th mean. * Almost sure representation. Usually, convergence in distribution does not imply convergence almost surely. However, for a given sequence which converges in distribution to ''X''0 it is always possible to find a new probability space (Ω, ''F'', P) and random variables defined on it such that ''Yn'' is equal in distribution to for each , and ''Yn'' converges to ''Y''0 almost surely. * If for all ''ε'' > 0, *::\sum_n \mathbb \left(, X_n - X, > \varepsilon\right) < \infty, *:then we say that ''converges almost completely'', or ''almost in probability'' towards ''X''. When converges almost completely towards ''X'' then it also converges almost surely to ''X''. In other words, if converges in probability to ''X'' sufficiently quickly (i.e. the above sequence of tail probabilities is summable for all ), then also converges almost surely to ''X''. This is a direct implication from the Borel–Cantelli lemma. * If is a sum of ''n'' real independent random variables: *::S_n = X_1+\cdots+X_n \, *:then converges almost surely if and only if converges in probability. * The dominated convergence theorem gives sufficient conditions for almost sure convergence to imply ''L''1-convergence: } *A necessary and sufficient condition for ''L''1 convergence is X_n\xrightarrow X and the sequence (''Xn'') is uniformly integrable. *If X_n are discrete and independent, then X_n \stackrel X implies that X_n \stackrel X. This is a consequence of the second Borel–Cantelli lemma.


See also

* Proofs of convergence of random variables * Convergence of measures * Convergence in measure * Continuous stochastic process: the question of continuity of a
stochastic process In probability theory and related fields, a stochastic () or random process is a mathematical object usually defined as a family of random variables. Stochastic processes are widely used as mathematical models of systems and phenomena that ap ...
is essentially a question of convergence, and many of the same concepts and relationships used above apply to the continuity question. *
Asymptotic distribution In mathematics and statistics, an asymptotic distribution is a probability distribution that is in a sense the "limiting" distribution of a sequence of distributions. One of the main uses of the idea of an asymptotic distribution is in providing ...
*
Big O in probability notation The order in probability notation is used in probability theory and statistical theory in direct parallel to the big-O notation that is standard in mathematics. Where the big-O notation deals with the convergence of sequences or sets of ordinary n ...
*
Skorokhod's representation theorem In mathematics and statistics, Skorokhod's representation theorem is a result that shows that a weakly convergent sequence of probability measures whose limit measure is sufficiently well-behaved can be represented as the distribution/law of a po ...
* The Tweedie convergence theorem *
Slutsky's theorem In probability theory, Slutsky’s theorem extends some properties of algebraic operations on convergent sequences of real numbers to sequences of random variables. The theorem was named after Eugen Slutsky. Slutsky's theorem is also attributed t ...
* Continuous mapping theorem


Notes


References

* * * * * * * * * * * * * * * https://www.ma.utexas.edu/users/gordanz/notes/weak.pdf {{DEFAULTSORT:Convergence Of Random Variables Stochastic processes Random variables, Convergence of>X_n, ^r= E X, ^r


Properties

Provided the probability space is
complete Complete may refer to: Logic * Completeness (logic) * Completeness of a theory, the property of a theory that every formula in the theory's language or its negation is provable Mathematics * The completeness of the real numbers, which implies t ...
: * If X_n\ \xrightarrow\ X and X_n\ \xrightarrow\ Y, then X=Y
almost surely In probability theory, an event is said to happen almost surely (sometimes abbreviated as a.s.) if it happens with probability 1 (or Lebesgue measure 1). In other words, the set of possible exceptions may be non-empty, but it has probability 0. ...
. * If X_n\ \xrightarrow\ X and X_n\ \xrightarrow\ Y, then X=Y almost surely. * If X_n\ \xrightarrow\ X and X_n\ \xrightarrow\ Y, then X=Y almost surely. * If X_n\ \xrightarrow\ X and Y_n\ \xrightarrow\ Y, then aX_n+bY_n\ \xrightarrow\ aX+bY (for any real numbers and ) and X_n Y_n\xrightarrow\ XY. * If X_n\ \xrightarrow\ X and Y_n\ \xrightarrow\ Y, then aX_n+bY_n\ \xrightarrow\ aX+bY (for any real numbers and ) and X_n Y_n\xrightarrow\ XY. * If X_n\ \xrightarrow\ X and Y_n\ \xrightarrow\ Y, then aX_n+bY_n\ \xrightarrow\ aX+bY (for any real numbers and ). * None of the above statements are true for convergence in distribution. The chain of implications between the various notions of convergence are noted in their respective sections. They are, using the arrow notation: : \begin \xrightarrow & \underset & \xrightarrow & & \\ & & \Downarrow & & \\ \xrightarrow & \Rightarrow & \xrightarrow & \Rightarrow & \xrightarrow \end These properties, together with a number of other special cases, are summarized in the following list: * Almost sure convergence implies convergence in probability: roof/sup> *:X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X * Convergence in probability implies there exists a sub-sequence (n_k) which almost surely converges: *: X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_\ \xrightarrow\ X * Convergence in probability implies convergence in distribution: roof/sup> *: X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X * Convergence in ''r''-th order mean implies convergence in probability: *: X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X * Convergence in ''r''-th order mean implies convergence in lower order mean, assuming that both orders are greater than or equal to one: *: X_n\ \xrightarrow\ X \quad\Rightarrow\quad X_n\ \xrightarrow\ X, provided ''r'' ≥ ''s'' ≥ 1. * If ''X''''n'' converges in distribution to a constant ''c'', then ''X''''n'' converges in probability to ''c'': roof/sup> *: X_n\ \xrightarrow\ c \quad\Rightarrow\quad X_n\ \xrightarrow\ c, provided ''c'' is a constant. * If converges in distribution to ''X'' and the difference between ''Xn'' and ''Yn'' converges in probability to zero, then ''Yn'' also converges in distribution to ''X'': roof/sup> *: X_n\ \xrightarrow\ X,\ \ , X_n-Y_n, \ \xrightarrow\ 0\ \quad\Rightarrow\quad Y_n\ \xrightarrow\ X * If converges in distribution to ''X'' and ''Yn'' converges in distribution to a constant ''c'', then the joint vector converges in distribution to : roof/sup> *: X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ c\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,c) provided ''c'' is a constant. *:Note that the condition that converges to a constant is important, if it were to converge to a random variable ''Y'' then we wouldn't be able to conclude that converges to . * If ''Xn'' converges in probability to ''X'' and ''Yn'' converges in probability to ''Y'', then the joint vector converges in probability to : roof/sup> *: X_n\ \xrightarrow\ X,\ \ Y_n\ \xrightarrow\ Y\ \quad\Rightarrow\quad (X_n,Y_n)\ \xrightarrow\ (X,Y) * If converges in probability to ''X'', and if for all ''n'' and some ''b'', then converges in ''r''th mean to ''X'' for all . In other words, if converges in probability to ''X'' and all random variables are almost surely bounded above and below, then converges to ''X'' also in any ''r''th mean. * Almost sure representation. Usually, convergence in distribution does not imply convergence almost surely. However, for a given sequence which converges in distribution to ''X''0 it is always possible to find a new probability space (Ω, ''F'', P) and random variables defined on it such that ''Yn'' is equal in distribution to for each , and ''Yn'' converges to ''Y''0 almost surely. * If for all ''ε'' > 0, *::\sum_n \mathbb \left(, X_n - X, > \varepsilon\right) < \infty, *:then we say that ''converges almost completely'', or ''almost in probability'' towards ''X''. When converges almost completely towards ''X'' then it also converges almost surely to ''X''. In other words, if converges in probability to ''X'' sufficiently quickly (i.e. the above sequence of tail probabilities is summable for all ), then also converges almost surely to ''X''. This is a direct implication from the Borel–Cantelli lemma. * If is a sum of ''n'' real independent random variables: *::S_n = X_1+\cdots+X_n \, *:then converges almost surely if and only if converges in probability. * The dominated convergence theorem gives sufficient conditions for almost sure convergence to imply ''L''1-convergence: } *A necessary and sufficient condition for ''L''1 convergence is X_n\xrightarrow X and the sequence (''Xn'') is uniformly integrable. *If X_n are discrete and independent, then X_n \stackrel X implies that X_n \stackrel X. This is a consequence of the second Borel–Cantelli lemma.


See also

* Proofs of convergence of random variables * Convergence of measures * Convergence in measure * Continuous stochastic process: the question of continuity of a
stochastic process In probability theory and related fields, a stochastic () or random process is a mathematical object usually defined as a family of random variables. Stochastic processes are widely used as mathematical models of systems and phenomena that ap ...
is essentially a question of convergence, and many of the same concepts and relationships used above apply to the continuity question. *
Asymptotic distribution In mathematics and statistics, an asymptotic distribution is a probability distribution that is in a sense the "limiting" distribution of a sequence of distributions. One of the main uses of the idea of an asymptotic distribution is in providing ...
*
Big O in probability notation The order in probability notation is used in probability theory and statistical theory in direct parallel to the big-O notation that is standard in mathematics. Where the big-O notation deals with the convergence of sequences or sets of ordinary n ...
*
Skorokhod's representation theorem In mathematics and statistics, Skorokhod's representation theorem is a result that shows that a weakly convergent sequence of probability measures whose limit measure is sufficiently well-behaved can be represented as the distribution/law of a po ...
* The Tweedie convergence theorem *
Slutsky's theorem In probability theory, Slutsky’s theorem extends some properties of algebraic operations on convergent sequences of real numbers to sequences of random variables. The theorem was named after Eugen Slutsky. Slutsky's theorem is also attributed t ...
* Continuous mapping theorem


Notes


References

* * * * * * * * * * * * * * * https://www.ma.utexas.edu/users/gordanz/notes/weak.pdf {{DEFAULTSORT:Convergence Of Random Variables Stochastic processes Random variables, Convergence of