In
mathematics, the Kronecker product, sometimes denoted by ⊗, is an
operation on two
matrices
Matrix most commonly refers to:
* ''The Matrix'' (franchise), an American media franchise
** ''The Matrix'', a 1999 science-fiction action film
** "The Matrix", a fictional setting, a virtual reality environment, within ''The Matrix'' (franchis ...
of arbitrary size resulting in a
block matrix. It is a generalization of the
outer product (which is denoted by the same symbol) from vectors to matrices, and gives the matrix of the
tensor product
In mathematics, the tensor product V \otimes W of two vector spaces and (over the same field) is a vector space to which is associated a bilinear map V\times W \to V\otimes W that maps a pair (v,w),\ v\in V, w\in W to an element of V \otime ...
linear map with respect to a standard choice of
basis
Basis may refer to:
Finance and accounting
* Adjusted basis, the net cost of an asset after adjusting for various tax-related items
*Basis point, 0.01%, often used in the context of interest rates
* Basis trading, a trading strategy consisting ...
. The Kronecker product is to be distinguished from the usual
matrix multiplication
In mathematics, particularly in linear algebra, matrix multiplication is a binary operation that produces a matrix from two matrices. For matrix multiplication, the number of columns in the first matrix must be equal to the number of rows in the s ...
, which is an entirely different operation. The Kronecker product is also sometimes called matrix direct product.
The Kronecker product is named after the German mathematician
Leopold Kronecker
Leopold Kronecker (; 7 December 1823 – 29 December 1891) was a German mathematician who worked on number theory, algebra and logic. He criticized Georg Cantor's work on set theory, and was quoted by as having said, "'" ("God made the integers, ...
(1823–1891), even though there is little evidence that he was the first to define and use it. The Kronecker product has also been called the ''Zehfuss matrix'', and the ''Zehfuss product'', after , who in 1858 described this matrix operation, but Kronecker product is currently the most widely used.
Definition
If A is an matrix and B is a matrix, then the Kronecker product is the block matrix:
:
more explicitly:
:
Using
and
to denote
truncating integer division and
remainder
In mathematics, the remainder is the amount "left over" after performing some computation. In arithmetic, the remainder is the integer "left over" after dividing one integer by another to produce an integer quotient ( integer division). In algeb ...
, respectively, and numbering the matrix elements starting from 0, one obtains
and
For the usual numbering starting from 1, one obtains
and
If A and B represent
linear transformations
In mathematics, and more specifically in linear algebra, a linear map (also called a linear mapping, linear transformation, vector space homomorphism, or in some contexts linear function) is a mapping V \to W between two vector spaces that pre ...
and , respectively, then represents the
tensor product
In mathematics, the tensor product V \otimes W of two vector spaces and (over the same field) is a vector space to which is associated a bilinear map V\times W \to V\otimes W that maps a pair (v,w),\ v\in V, w\in W to an element of V \otime ...
of the two maps, .
Examples
:
Similarly:
:
Properties
Relations to other matrix operations
) = \exp(\mathbf) \otimes \exp(\mathbf)
Kronecker sums appear naturally in
physics
Physics is the natural science that studies matter, its fundamental constituents, its motion and behavior through space and time, and the related entities of energy and force. "Physical science is that department of knowledge which r ...
when considering ensembles of non-interacting
systems. Let ''H
k'' be the Hamiltonian of the ''k''th such system. Then the total Hamiltonian of the ensemble is
:
Abstract properties
Matrix equations
The Kronecker product can be used to get a convenient representation for some matrix equations. Consider for instance the equation , where A, B and C are given matrices and the matrix X is the unknown. We can use the "vec trick" to rewrite this equation as
:
Here, vec(X) denotes the
vectorization
Vectorization may refer to:
Computing
* Array programming, a style of computer programming where operations are applied to whole arrays instead of individual elements
* Automatic vectorization, a compiler optimization that transforms loops to vec ...
of the matrix X, formed by stacking the columns of X into a single
column vector
In linear algebra, a column vector with m elements is an m \times 1 matrix consisting of a single column of m entries, for example,
\boldsymbol = \begin x_1 \\ x_2 \\ \vdots \\ x_m \end.
Similarly, a row vector is a 1 \times n matrix for some n, c ...
.
It now follows from the properties of the Kronecker product that the equation has a unique solution, if and only if A and B are invertible .
If X and C are row-ordered into the column vectors u and v, respectively, then
:
The reason is that
:
Applications
For an example of the application of this formula, see the article on the
Lyapunov equation In control theory, the discrete Lyapunov equation is of the form
:A X A^ - X + Q = 0
where Q is a Hermitian matrix and A^H is the conjugate transpose of A.
The continuous Lyapunov equation is of the form
:AX + XA^H + Q = 0.
The Lyapunov equation o ...
. This formula also comes in handy in showing that the
matrix normal distribution
In statistics, the matrix normal distribution or matrix Gaussian distribution is a probability distribution that is a generalization of the multivariate normal distribution to matrix-valued random variables.
Definition
The probability density ...
is a special case of the
multivariate normal distribution
In probability theory and statistics, the multivariate normal distribution, multivariate Gaussian distribution, or joint normal distribution is a generalization of the one-dimensional ( univariate) normal distribution to higher dimensions. One ...
. This formula is also useful for representing 2D
image processing operations in matrix-vector form.
Another example is when a matrix can be factored as a
Hadamard product, then matrix multiplication can be performed faster by using the above formula. This can be applied recursively, as done in the
radix-2 FFT and the
Fast Walsh–Hadamard transform. Splitting a known matrix into the Hadamard product of two smaller matrices is known as the "nearest Kronecker product" problem, and can be solved exactly
[
] by using the
SVD
''Svenska Dagbladet'' (, "The Swedish Daily News"), abbreviated SvD, is a daily newspaper published in Stockholm, Sweden.
History and profile
The first issue of ''Svenska Dagbladet'' appeared on 18 December 1884. During the beginning of the ...
. To split a matrix into the Hadamard product of more than two matrices, in an optimal fashion, is a difficult problem and the subject of ongoing research; some authors cast it as a tensor decomposition problem.
In conjunction with the
least squares method, the Kronecker product can be used as an accurate solution to the
hand eye calibration problem
In robotics and mathematics, the hand eye calibration problem (also called the robot-sensor or robot-world calibration problem) is the problem of determining the transformation between a robot end-effector and a sensor or sensors (camera or laser s ...
.
Related matrix operations
Two related matrix operations are the Tracy–Singh and
Khatri–Rao product
In mathematics, the Khatri–Rao product of matrices defined as
: \mathbf \ast \mathbf = \left(\mathbf_ \otimes \mathbf_\right)_
in which the ''ij''-th block is the sized Kronecker product of the corresponding blocks of A and B, assuming the numb ...
s, which operate on
partitioned matrices. Let the matrix A be partitioned into the blocks A
''ij'' and matrix B into the blocks B
''kl'', with of course , , and .
Tracy–Singh product
The Tracy–Singh product is defined as
[
]
:
which means that the (''ij'')-th subblock of the product is the matrix , of which the (''k'')-th subblock equals the matrix . Essentially the Tracy–Singh product is the pairwise Kronecker product for each pair of partitions in the two matrices.
For example, if A and B both are partitioned matrices e.g.:
:
we get:
:
Khatri–Rao product
* Block Kronecker product
* Column-wise Khatri–Rao product
Face-splitting product
Mixed-products properties
[
]
:
where
denotes the
Khatri–Rao product#Face-splitting product, Face-splitting product.
[
][
]
:
Similarly:
[
]
:
:
where
and
are
vector
Vector most often refers to:
*Euclidean vector, a quantity with a magnitude and a direction
*Vector (epidemiology), an agent that carries and transmits an infectious pathogen into another living organism
Vector may also refer to:
Mathematic ...
s,
[
]
:
where
and
are
vector
Vector most often refers to:
*Euclidean vector, a quantity with a magnitude and a direction
*Vector (epidemiology), an agent that carries and transmits an infectious pathogen into another living organism
Vector may also refer to:
Mathematic ...
s, and
denotes the
Hadamard product.
Similarly:
:
:
,
where
is vector
convolution
In mathematics (in particular, functional analysis), convolution is a mathematical operation on two functions ( and ) that produces a third function (f*g) that expresses how the shape of one is modified by the other. The term ''convolution'' ...
and
is the
Fourier transform matrix (this result is an evolving of
count sketch
Count sketch is a type of dimensionality reduction that is particularly efficient in statistics, machine learning and algorithms.
It was invented by
Moses Charikar, Kevin Chen and Martin Farach-Colton in an effort to speed up the AMS Sketch by ...
properties
[
]),
:
where
denotes the
Column-wise Khatri–Rao product.
Similarly:
:
:
where
and
are
vector
Vector most often refers to:
*Euclidean vector, a quantity with a magnitude and a direction
*Vector (epidemiology), an agent that carries and transmits an infectious pathogen into another living organism
Vector may also refer to:
Mathematic ...
s.
See also
*
Generalized linear array model
*
Kronecker coefficient In mathematics, Kronecker coefficients ''g''λ''μν'' describe the decomposition of the tensor product (= Kronecker product) of two irreducible representations of a symmetric group into irreducible representations. They play an important role ...
Notes
References
*
*
*
*
*
External links
*
*
*
*
* The entry on the Kronecker, Zehfuss, or Direct Product of matrices has historical information.
*
* Software source in more than 40 languages.
{{DEFAULTSORT:Kronecker Product
Matrix theory