In
computer vision
Computer vision tasks include methods for image sensor, acquiring, Image processing, processing, Image analysis, analyzing, and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical ...
, the fundamental matrix
is a 3×3
matrix
Matrix (: matrices or matrixes) or MATRIX may refer to:
Science and mathematics
* Matrix (mathematics), a rectangular array of numbers, symbols or expressions
* Matrix (logic), part of a formula in prenex normal form
* Matrix (biology), the m ...
which relates corresponding points in
stereo images. In
epipolar geometry
Epipolar geometry is the geometry of stereo vision#Computer stereo vision, stereo vision. When two cameras view a 3D scene from two distinct positions, there are a number of geometric relations between the 3D points and their projections onto th ...
, with
homogeneous image coordinates, x and x′, of corresponding points in a stereo image pair, Fx describes a line (an
epipolar line) on which the corresponding point x′ on the other image must lie. That means, for all pairs of corresponding points holds
:
Being of rank two and determined only up to scale, the fundamental matrix can be estimated given at least seven point correspondences. Its seven parameters represent the only geometric information about cameras that can be obtained through point correspondences alone.
The term "fundamental matrix" was coined by
QT Luong in his influential PhD thesis. It is sometimes also referred to as the "bifocal tensor". As a tensor it is a
two-point tensor in that it is a
bilinear form
In mathematics, a bilinear form is a bilinear map on a vector space (the elements of which are called '' vectors'') over a field ''K'' (the elements of which are called '' scalars''). In other words, a bilinear form is a function that is linea ...
relating points in distinct coordinate systems.
The above relation which defines the fundamental matrix was published in 1992 by both
Olivier Faugeras and
Richard Hartley. Although
H. Christopher Longuet-Higgins'
essential matrix satisfies a similar relationship, the essential matrix is a metric object pertaining to calibrated cameras, while the fundamental matrix describes the correspondence in more general and fundamental terms of projective geometry.
This is captured mathematically by the relationship between a fundamental matrix
and its corresponding essential matrix
,
which is
:
and
being the intrinsic calibration
matrices of the two images involved.
Introduction
The fundamental matrix is a relationship between any two images of the same scene that constrains where the projection of points from the scene can occur in both images. Given the projection of a scene point into one of the images the corresponding point in the other image is constrained to a line, helping the search, and allowing for the detection of wrong correspondences. The relation between
corresponding points, which the fundamental matrix represents, is referred to as ''epipolar constraint'', ''matching constraint'', ''discrete matching constraint'', or ''incidence relation''.
Projective reconstruction theorem
The fundamental matrix can be determined by a set of
point correspondences. Additionally, these corresponding image points may be ''triangulated'' to world points with the help of camera matrices derived directly from this fundamental matrix. The scene composed of these world points is within a
projective transformation
In projective geometry, a homography is an isomorphism of projective spaces, induced by an isomorphism of the vector spaces from which the projective spaces derive. It is a bijection that maps lines to lines, and thus a collineation. In general, ...
of the true scene.
Proof
Say that the image point correspondence
derives from the world point
under the camera matrices
as
:
Say we transform space by a general
homography matrix
such that
.
The cameras then transform as
:
:
and likewise with
still get us the same image points.
Derivation of the fundamental matrix using coplanarity condition
The fundamental matrix can also be derived using the coplanarity condition.
[Jaehong Oh]
"Novel Approach to Epipolar Resampling of HRSI and Satellite Stereo Imagery-based Georeferencing of Aerial Images"
, 2011, pp. 22–29 accessed 2011-08-05.
For satellite images
The fundamental matrix expresses the epipolar geometry in stereo images. The
epipolar geometry
Epipolar geometry is the geometry of stereo vision#Computer stereo vision, stereo vision. When two cameras view a 3D scene from two distinct positions, there are a number of geometric relations between the 3D points and their projections onto th ...
in images taken with perspective cameras appears as straight lines. However, in
satellite images, the image is formed during the sensor movement along its orbit (
pushbroom sensor). Therefore, there are multiple projection centers for one image scene and the epipolar line is formed as an epipolar curve. However, in special conditions such as small image tiles, the satellite images could be rectified using the fundamental matrix.
Properties
The fundamental matrix is of
rank 2. Its
kernel defines the
epipole.
See also
*
Epipolar geometry
Epipolar geometry is the geometry of stereo vision#Computer stereo vision, stereo vision. When two cameras view a 3D scene from two distinct positions, there are a number of geometric relations between the 3D points and their projections onto th ...
*
Essential matrix
*
Trifocal tensor
In computer vision, the trifocal tensor (also tritensor) is a 3×3×3 array of numbers (i.e., a tensor) that incorporates all projective geometric relationships among three views. It relates the coordinates of corresponding points or lines in thre ...
*
Eight-point algorithm The eight-point algorithm is an algorithm used in computer vision to estimate the essential matrix or the Fundamental matrix (computer vision), fundamental matrix related to a stereo camera pair from a set of corresponding image points. It was intr ...
Notes
References
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
Toolboxes
fundestis a
GPL C/
C++ library for
robust, non-linear (based on the
Levenberg–Marquardt algorithm) fundamental matrix estimation from matched point pairs and various objective functions (Manolis Lourakis).
Structure and Motion Toolkit in MATLAB (Philip H. S. Torr)The Epipolar Geometry Toolbox (EGT)
External links
Epipolar Geometry and the Fundamental Matrix (chapter from Hartley & Zisserman)*
ttps://web.archive.org/web/20091120063117/http://www2.informatik.hu-berlin.de/~blaschek/diplvortrag/learn_epi/EpipolarGeo.html Visualization of epipolar geometry(originally by Sylvain Bougnoux of
INRIA
The National Institute for Research in Digital Science and Technology (Inria) () is a French national research institution focusing on computer science and applied mathematics.
It was created under the name French Institute for Research in Comp ...
Robotvis, requires
Java
Java is one of the Greater Sunda Islands in Indonesia. It is bordered by the Indian Ocean to the south and the Java Sea (a part of Pacific Ocean) to the north. With a population of 156.9 million people (including Madura) in mid 2024, proje ...
)
The Fundamental Matrix SongVideo demonstrating laws of epipolar geometry.
{{Matrix classes
Geometry in computer vision