Cross-covariance Matrix

In probability theory and statistics, a cross-covariance matrix is a matrix whose element in the ''i'', ''j'' position is the covariance between the ''i''-th element of a random vector and ''j''-th element of another random vector.
When the two random vectors are the same, the cross-covariance matrix is referred to as covariance matrix.
A random vector is a random variable with multiple dimensions. Each element of the vector is a scalar random variable. Each element has either a finite number of ''observed'' empirical values or a finite or infinite number of ''potential'' values. The potential values are specified by a theoretical joint probability distribution. Intuitively, the cross-covariance matrix generalizes the notion of covariance to multiple dimensions.

The cross-covariance matrix of two random vectors $\mathbf$ and $\mathbf$ is typically denoted by $\operatorname_$ or $\Sigma_$.

Definition

For random vectors $\mathbf$ and $\mathbf$, each containing random elements whose expected value and variance exist, the cross-covariance matrix of $\mathbf$ and $\mathbf$ is defined by

where \mathbf = \operatorname mathbf/math> and \mathbf = \operatorname mathbf/math> are vectors containing the expected values of $\mathbf$ and $\mathbf$. The vectors $\mathbf$ and $\mathbf$ need not have the same dimension, and either might be a scalar value.

The cross-covariance matrix is the matrix whose $(i,j)$ entry is the covariance

:\operatorname_ = \operatorname _i, Y_j= \operatorname X_i - \operatorname[X_i(Y_j - \operatorname[Y_j">_i.html" ;"title="X_i - \operatorname[X_i">X_i - \operatorname[X_i(Y_j - \operatorname[Y_j]

between the ''i''-th element of $\mathbf$ and the ''j''-th element of $\mathbf$. This gives the following component-wise definition of the cross-covariance matrix.

:
\operatorname_=
\begin
\mathrm X_1 - \operatorname[X_1(Y_1 - \operatorname[Y_1">_1.html" ;"title="X_1 - \operatorname[X_1">X_1 - \operatorname[X_1(Y_1 - \operatorname[Y_1] & \mathrm X_1 - \operatorname[X_1(Y_2 - \operatorname[Y_2])] & \cdots & \mathrm X_1 - \operatorname[X_1(Y_n - \operatorname[Y_n">_1.html" ;"title="X_1 - \operatorname[X_1">X_1 - \operatorname[X_1(Y_n - \operatorname[Y_n] \\ \\
\mathrm[(X_2 - \operatorname[X_2])(Y_1 - \operatorname[Y_1])] & \mathrm[(X_2 - \operatorname[X_2])(Y_2 - \operatorname[Y_2])] & \cdots & \mathrm[(X_2 - \operatorname[X_2])(Y_n - \operatorname _n] \\ \\
\vdots & \vdots & \ddots & \vdots \\ \\
\mathrm X_m - \operatorname[X_m(Y_1 - \operatorname[Y_1">_m.html" ;"title="X_m - \operatorname[X_m">X_m - \operatorname[X_m(Y_1 - \operatorname[Y_1] & \mathrm X_m - \operatorname[X_m(Y_2 - \operatorname[Y_2])] & \cdots & \mathrm X_m - \operatorname[X_m(Y_n - \operatorname _n">_m.html" ;"title="X_m - \operatorname[X_m">X_m - \operatorname[X_m(Y_n - \operatorname _n\end

Example

For example, if $\mathbf = \left( X_1,X_2,X_3 \right)^$ and $\mathbf = \left( Y_1,Y_2 \right)^$ are random vectors, then
$\operatorname(\mathbf,\mathbf)$ is a $3 \times 2$ matrix whose $(i,j)$-th entry is $\operatorname(X_i,Y_j)$.

Properties

For the cross-covariance matrix, the following basic properties apply:

# $\operatorname(\mathbf,\mathbf) = \operatorname[\mathbf \mathbf^] - \mathbf \mathbf^$
# $\operatorname(\mathbf,\mathbf) = \operatorname(\mathbf,\mathbf)^$
# $\operatorname(\mathbf + \mathbf,\mathbf) = \operatorname(\mathbf,\mathbf) + \operatorname(\mathbf, \mathbf)$
# $\operatorname(A\mathbf+ \mathbf, B^\mathbf + \mathbf) = A\, \operatorname(\mathbf, \mathbf) \,B$
# If $\mathbf$ and $\mathbf$ are independent (or somewhat less restrictedly, if every random variable in $\mathbf$ is uncorrelated with every random variable in $\mathbf$), then $\operatorname(\mathbf,\mathbf) = 0_$

where $\mathbf$, $\mathbf$ and $\mathbf$ are random $p \times 1$ vectors, $\mathbf$ is a random $q \times 1$ vector, $\mathbf$ is a $q \times 1$ vector, $\mathbf$ is a $p \times 1$ vector, $A$ and $B$ are $q \times p$ matrices of constants, and $0_$ is a $p \times q$ matrix of zeroes.

Definition for complex random vectors

If $\mathbf$ and $\mathbf$ are complex random vectors, the definition of the cross-covariance matrix is slightly changed. Transposition is replaced by Hermitian transposition:

:\operatorname_ = \operatorname(\mathbf,\mathbf) \stackrel\ \operatorname \mathbf-\mathbf)(\mathbf-\mathbf)^/math>

For complex random vectors, another matrix called the pseudo-cross-covariance matrix is defined as follows:

:\operatorname_ = \operatorname(\mathbf,\overline) \stackrel\ \operatorname \mathbf-\mathbf)(\mathbf-\mathbf)^/math>

Uncorrelatedness

Two random vectors $\mathbf$ and $\mathbf$ are called uncorrelated if their cross-covariance matrix $\operatorname_$ matrix is a zero matrix.

Complex random vectors $\mathbf$ and $\mathbf$ are called uncorrelated if their covariance matrix and pseudo-covariance matrix is zero, i.e. if $\operatorname_ = \operatorname_ = 0$.

References

{{reflist

Covariance and correlation
Matrices (mathematics)