In
linear algebra
Linear algebra is the branch of mathematics concerning linear equations such as:
:a_1x_1+\cdots +a_nx_n=b,
linear maps such as:
:(x_1, \ldots, x_n) \mapsto a_1x_1+\cdots +a_nx_n,
and their representations in vector spaces and through matric ...
, orthogonalization is the process of finding a set of
orthogonal vector
In mathematics, an inner product space (or, rarely, a Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation called an inner product. The inner product of two vectors in the space is a scalar, often den ...
s that
span
Span may refer to:
Science, technology and engineering
* Span (unit), the width of a human hand
* Span (engineering), a section between two intermediate supports
* Wingspan, the distance between the wingtips of a bird or aircraft
* Sorbitan es ...
a particular
subspace. Formally, starting with a
linearly independent
In the theory of vector spaces, a set of vectors is said to be if there is a nontrivial linear combination of the vectors that equals the zero vector. If no such linear combination exists, then the vectors are said to be . These concepts ...
set of vectors in an
inner product space
In mathematics, an inner product space (or, rarely, a Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation called an inner product. The inner product of two vectors in the space is a scalar, often ...
(most commonly the
Euclidean space
Euclidean space is the fundamental space of geometry, intended to represent physical space. Originally, that is, in Euclid's ''Elements'', it was the three-dimensional space of Euclidean geometry, but in modern mathematics there are Euclidean sp ...
R
''n''), orthogonalization results in a set of
orthogonal
In mathematics, orthogonality is the generalization of the geometric notion of '' perpendicularity''.
By extension, orthogonality is also used to refer to the separation of specific features of a system. The term also has specialized meanings in ...
vectors that generate the same subspace as the vectors ''v''
1, ... , ''v''
''k''. Every vector in the new set is orthogonal to every other vector in the new set; and the new set and the old set have the same
linear span
In mathematics, the linear span (also called the linear hull or just span) of a set of vectors (from a vector space), denoted , pp. 29-30, §§ 2.5, 2.8 is defined as the set of all linear combinations of the vectors in . It can be characteri ...
.
In addition, if we want the resulting vectors to all be
unit vector
In mathematics, a unit vector in a normed vector space is a vector (often a spatial vector) of length 1. A unit vector is often denoted by a lowercase letter with a circumflex, or "hat", as in \hat (pronounced "v-hat").
The term ''direction ve ...
s, then we
normalize each vector and the procedure is called orthonormalization.
Orthogonalization is also possible with respect to any
symmetric bilinear form In mathematics, a symmetric bilinear form on a vector space is a bilinear map from two copies of the vector space to the field of scalars such that the order of the two vectors does not affect the value of the map. In other words, it is a biline ...
(not necessarily an inner product, not necessarily over
real number
In mathematics, a real number is a number that can be used to measurement, measure a ''continuous'' one-dimensional quantity such as a distance, time, duration or temperature. Here, ''continuous'' means that values can have arbitrarily small var ...
s), but standard algorithms may encounter
division by zero
In mathematics, division by zero is division where the divisor (denominator) is zero. Such a division can be formally expressed as \tfrac, where is the dividend (numerator). In ordinary arithmetic, the expression has no meaning, as there is ...
in this more general setting.
Orthogonalization algorithms
Methods for performing orthogonalization include:
*
Gram–Schmidt process
In mathematics, particularly linear algebra and numerical analysis, the Gram–Schmidt process is a method for orthonormalizing a set of vectors in an inner product space, most commonly the Euclidean space equipped with the standard inner prod ...
, which uses
projection
*
Householder transformation
In linear algebra, a Householder transformation (also known as a Householder reflection or elementary reflector) is a linear transformation that describes a reflection about a plane or hyperplane containing the origin. The Householder transformat ...
, which uses
reflection
*
Givens rotation In numerical linear algebra, a Givens rotation is a rotation in the plane spanned by two coordinates axes. Givens rotations are named after Wallace Givens, who introduced them to numerical analysts in the 1950s while he was working at Argonne Nation ...
*Symmetric orthogonalization, which uses the
Singular value decomposition
In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix. It generalizes the eigendecomposition of a square normal matrix with an orthonormal eigenbasis to any \ m \times n\ matrix. It is r ...
When performing orthogonalization on a computer, the Householder transformation is usually preferred over the Gram–Schmidt process since it is more
numerically stable
In the mathematical subfield of numerical analysis, numerical stability is a generally desirable property of numerical algorithms. The precise definition of stability depends on the context. One is numerical linear algebra and the other is algor ...
, i.e. rounding errors tend to have less serious effects.
On the other hand, the Gram–Schmidt process produces the jth orthogonalized vector after the jth iteration, while orthogonalization using Householder reflections produces all the vectors only at the end. This makes only the Gram–Schmidt process applicable for
iterative method
In computational mathematics, an iterative method is a mathematical procedure that uses an initial value to generate a sequence of improving approximate solutions for a class of problems, in which the ''n''-th approximation is derived from the pre ...
s like the
Arnoldi iteration In numerical linear algebra, the Arnoldi iteration is an eigenvalue algorithm and an important example of an iterative method. Arnoldi finds an approximation to the eigenvalues and eigenvectors of general (possibly non-Hermitian) matrices by con ...
.
The Givens rotation is more easily
parallelized
Parallel computing is a type of computation in which many calculations or processes are carried out simultaneously. Large problems can often be divided into smaller ones, which can then be solved at the same time. There are several different for ...
than Householder transformations.
Symmetric orthogonalization was formulated by
Per-Olov Löwdin
Per-Olov Löwdin (October 28, 1916 – October 6, 2000) was a Swedish physicist, professor at the University of Uppsala from 1960 to 1983, and in parallel at the University of Florida until 1993.
A former graduate student under Ivar Waller, Löw ...
.
Local orthogonalization
To compensate for the loss of useful signal in traditional noise attenuation approaches because of incorrect parameter selection or inadequacy of denoising assumptions, a weighting operator can be applied on the initially denoised section for the retrieval of useful signal from the initial noise section. The new denoising process is referred to as the local orthogonalization of signal and noise.
It has a wide range of applications in many signals processing and seismic exploration fields.
See also
{{wiktionary, orthogonalization
*
Orthogonality
In mathematics, orthogonality is the generalization of the geometric notion of '' perpendicularity''.
By extension, orthogonality is also used to refer to the separation of specific features of a system. The term also has specialized meanings i ...
*
Biorthogonal system In mathematics, a biorthogonal system is a pair of indexed families of vectors
\tilde v_i \text E \text \tilde u_i \text F
such that
\left\langle\tilde v_i , \tilde u_j\right\rangle = \delta_,
where E and F form a pair of topological vector spaces ...
*
Orthogonal basis In mathematics, particularly linear algebra, an orthogonal basis for an inner product space V is a basis for V whose vectors are mutually orthogonal. If the vectors of an orthogonal basis are normalized, the resulting basis is an orthonormal b ...
References
Linear algebra