In
probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set ...
and
statistics, the mathematical concepts of covariance and correlation are very similar.
Both describe the degree to which two
random variables or
sets of random variables tend to deviate from their
expected values in similar ways.
If ''X'' and ''Y'' are two random variables, with
mean
There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value (magnitude and sign) of a given data set.
For a data set, the '' ari ...
s (expected values) ''μ
X'' and ''μ
Y'' and
standard deviations ''σ
X'' and ''σ
Y'', respectively, then their covariance and correlation are as follows:
:
so that
:
where ''E'' is the expected value operator. Notably, correlation is
dimensionless
A dimensionless quantity (also known as a bare quantity, pure quantity, or scalar quantity as well as quantity of dimension one) is a quantity to which no physical dimension is assigned, with a corresponding SI unit of measurement of one (or 1) ...
while covariance is in units obtained by multiplying the units of the two variables.
If ''Y'' always takes on the same values as ''X'', we have the covariance of a variable with itself (i.e.
), which is called the
variance
In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbe ...
and is more commonly denoted as
the square of the standard deviation. The ''correlation'' of a variable with itself is always 1 (except in the
degenerate case where the two variances are zero because ''X'' always takes on the same single value, in which case the correlation does not exist since its computation would involve
division by 0). More generally, the correlation between two variables is 1 (or –1) if one of them always takes on a value that is given exactly by a
linear function
In mathematics, the term linear function refers to two distinct but related notions:
* In calculus and related areas, a linear function is a function whose graph is a straight line, that is, a polynomial function of degree zero or one. For dist ...
of the other with respectively a positive (or negative)
slope
In mathematics, the slope or gradient of a line is a number that describes both the ''direction'' and the ''steepness'' of the line. Slope is often denoted by the letter ''m''; there is no clear answer to the question why the letter ''m'' is use ...
.
Although the values of the theoretical covariances and correlations are linked in the above way, the probability distributions of
sample estimates of these quantities are not linked in any simple way and they generally need to be treated separately.
Multiple random variables
With any number of random variables in excess of 1, the variables can be stacked into a
random vector
In probability, and statistics, a multivariate random variable or random vector is a list of mathematical variables each of whose value is unknown, either because the value has not yet occurred or because there is imperfect knowledge of its value ...
whose ''i''
th element is the ''i''
th random variable. Then the variances and covariances can be placed in a
covariance matrix, in which the (''i,j'') element is the covariance between the ''i''
th random variable and the ''j''
th one. Likewise, the correlations can be placed in a
correlation matrix.
Time series analysis
In the case of a
time series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Ex ...
which is
stationary in the wide sense, both the means and variances are constant over time (E(''X
n+m'') = E(''X
n'') ='' μ
X'' and var(''X
n+m'') = var(''X
n'') and likewise for the variable ''Y''). In this case the cross-covariance and cross-correlation are functions of the time difference:
:
If ''Y'' is the same variable as ''X'', the above expressions are called the ''autocovariance'' and ''autocorrelation'':
:
References
{{Refimprove, date=August 2011