In mathematics, the Hilbert projection theorem is a famous result of
convex analysis
Convex analysis is the branch of mathematics devoted to the study of properties of convex functions and convex sets, often with applications in convex minimization, a subdomain of optimization theory.
Convex sets
A subset C \subseteq X of som ...
that says that for every vector
in a
Hilbert space
In mathematics, Hilbert spaces (named after David Hilbert) allow generalizing the methods of linear algebra and calculus from (finite-dimensional) Euclidean vector spaces to spaces that may be infinite-dimensional. Hilbert spaces arise natural ...
and every nonempty closed convex
there exists a unique vector
for which
is minimized over the vectors
; that is, such that
for every
Finite dimensional case
Some intuition for the theorem can be obtained by considering the
first order condition
In calculus, a derivative test uses the derivatives of a function to locate the critical points of a function and determine whether each point is a local maximum, a local minimum, or a saddle point. Derivative tests can also give information about ...
of the optimization problem.
Consider a finite dimensional real Hilbert space
with a subspace
and a point
If
is a or of the function
defined by
(which is the same as the minimum point of
), then derivative must be zero at
In matrix derivative notation
Since
is a vector in
that represents an arbitrary tangent direction, it follows that
must be orthogonal to every vector in
Statement
Detailed elementary proof
Proof by reduction to a special case
It suffices to prove the theorem in the case of
because the general case follows from the statement below by replacing
with
Consequences
:
If
then
which implies
:
Let
where
is the underlying scalar field of
and define
which is continuous and linear because this is true of each of its coordinates
The set
is closed in
because
is closed in
and
is continuous.
The kernel of any linear map is a vector subspace of its domain, which is why
is a vector subspace of
:
Let
The Hilbert projection theorem guarantees the existence of a unique
such that
(or equivalently, for all
).
Let
so that
and it remains to show that
The inequality above can be rewritten as:
Because
and
is a vector space,
and
which implies that
The previous inequality thus becomes
or equivalently,
But this last statement is true if and only if
every
Thus
Properties
Expression as a global minimum
The statement and conclusion of the Hilbert projection theorem can be expressed in terms of global minimums of the followings functions. Their notation will also be used to simplify certain statements.
Given a non-empty subset
and some
define a function
A of
if one exists, is any point
in
such that
in which case
is equal to the of the function
which is:
Effects of translations and scalings
When this global minimum point
exists and is unique then denote it by
explicitly, the defining properties of
(if it exists) are:
The Hilbert projection theorem guarantees that this unique minimum point exists whenever
is a non-empty closed and convex subset of a Hilbert space.
However, such a minimum point can also exist in non-convex or non-closed subsets as well; for instance, just as long is
is non-empty, if
then
If
is a non-empty subset,
is any scalar, and
are any vectors then
which implies:
Examples
The following counter-example demonstrates a continuous linear isomorphism
for which
Endow
with the dot product, let
and for every real
let
be the line of slope
through the origin, where it is readily verified that
Pick a real number
and define
by
(so this map scales the
coordinate by
while leaving the
coordinate unchanged).
Then
is an invertible continuous linear operator that satisfies
and
so that
and
Consequently, if
with
and if
then
See also
*
*
*
*
Notes
References
Bibliography
*
*
{{DEFAULTSORT:Hilbert Projection Theorem
Convex analysis
Theorems in functional analysis