HOME

TheInfoList



OR:

In mathematics, the structure
tensor In mathematics, a tensor is an algebraic object that describes a multilinear relationship between sets of algebraic objects related to a vector space. Tensors may map between different objects such as vectors, scalars, and even other tenso ...
, also referred to as the second-moment matrix, is a
matrix Matrix most commonly refers to: * ''The Matrix'' (franchise), an American media franchise ** ''The Matrix'', a 1999 science-fiction action film ** "The Matrix", a fictional setting, a virtual reality environment, within ''The Matrix'' (franchis ...
derived from the
gradient In vector calculus, the gradient of a scalar-valued differentiable function of several variables is the vector field (or vector-valued function) \nabla f whose value at a point p is the "direction and rate of fastest increase". If the gradi ...
of a function. It describes the distribution of the gradient in a specified neighborhood around a point and makes the information invariant respect the observing coordinates. The structure tensor is often used in
image processing An image is a visual representation of something. It can be two-dimensional, three-dimensional, or somehow otherwise feed into the visual system to convey information. An image can be an artifact, such as a photograph or other two-dimension ...
and
computer vision Computer vision is an interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the human ...
. J. Bigun and G. Granlund (1986), ''Optimal Orientation Detection of Linear Symmetry''. Tech. Report LiTH-ISY-I-0828, Computer Vision Laboratory, Linkoping University, Sweden 1986; Thesis Report, Linkoping studies in science and technology No. 85, 1986.


The 2D structure tensor


Continuous version

For a function I of two variables , the structure tensor is the 2×2 matrix : S_w(p) = \begin \int w(r) (I_x(p-r))^2\,d r & \int w(r) I_x(p-r)I_y(p-r)\,d r \\
0pt PT, Pt, or pt may refer to: Arts and entertainment * ''P.T.'' (video game), acronym for ''Playable Teaser'', a short video game released to promote the cancelled video game ''Silent Hills'' * Porcupine Tree, a British progressive rock group ...
\int w(r) I_x(p-r)I_y(p-r)\,d r & \int w(r) (I_y(p-r))^2\,d r \end where I_x and I_y are the
partial derivative In mathematics, a partial derivative of a function of several variables is its derivative with respect to one of those variables, with the others held constant (as opposed to the total derivative, in which all variables are allowed to vary). Par ...
s of I with respect to ''x'' and ''y''; the integrals range over the plane \mathbb^2; and ''w'' is some fixed "window function" (such as a
Gaussian blur In image processing, a Gaussian blur (also known as Gaussian smoothing) is the result of blurring an image by a Gaussian function (named after mathematician and scientist Carl Friedrich Gauss). It is a widely used effect in graphics software, ...
), a
distribution Distribution may refer to: Mathematics *Distribution (mathematics), generalized functions used to formulate solutions of partial differential equations *Probability distribution, the probability of a particular value or value range of a varia ...
on two variables. Note that the matrix S_w is itself a function of . The formula above can be written also as S_w(p)=\int w(r) S_0(p-r)\,d r, where S_0 is the matrix-valued function defined by : S_0(p)= \begin (I_x(p))^2 & I_x(p)I_y(p) \\
0pt PT, Pt, or pt may refer to: Arts and entertainment * ''P.T.'' (video game), acronym for ''Playable Teaser'', a short video game released to promote the cancelled video game ''Silent Hills'' * Porcupine Tree, a British progressive rock group ...
I_x(p)I_y(p) & (I_y(p))^2 \end If the
gradient In vector calculus, the gradient of a scalar-valued differentiable function of several variables is the vector field (or vector-valued function) \nabla f whose value at a point p is the "direction and rate of fastest increase". If the gradi ...
\nabla I = (I_x,I_y)^\text of I is viewed as a 2×1 (single-column) matrix, where (.)^\text denotes
transpose In linear algebra, the transpose of a matrix is an operator which flips a matrix over its diagonal; that is, it switches the row and column indices of the matrix by producing another matrix, often denoted by (among other notations). The tr ...
operation, turning a row vector to a column vector, the matrix S_0 can be written as the
matrix product In mathematics, particularly in linear algebra, matrix multiplication is a binary operation that produces a matrix from two matrices. For matrix multiplication, the number of columns in the first matrix must be equal to the number of rows in the ...
(\nabla I)(\nabla I)^\text or tensor or outer product \nabla I \otimes \nabla I. Note however that the structure tensor S_w(p) cannot be factored in this way in general except if w is a
Dirac delta function In mathematics, the Dirac delta distribution ( distribution), also known as the unit impulse, is a generalized function or distribution over the real numbers, whose value is zero everywhere except at zero, and whose integral over the entire ...
.


Discrete version

In image processing and other similar applications, the function I is usually given as a discrete
array An array is a systematic arrangement of similar objects, usually in rows and columns. Things called an array include: {{TOC right Music * In twelve-tone and serial composition, the presentation of simultaneous twelve-tone sets such that the ...
of samples I /math>, where ''p'' is a pair of integer indices. The 2D structure tensor at a given
pixel In digital imaging, a pixel (abbreviated px), pel, or picture element is the smallest addressable element in a raster image, or the smallest point in an all points addressable display device. In most digital display devices, pixels are the sm ...
is usually taken to be the discrete sum : S_w = \begin \sum_r w (I_x -r^2 & \sum_r w I_x -r_y -r\\
0pt PT, Pt, or pt may refer to: Arts and entertainment * ''P.T.'' (video game), acronym for ''Playable Teaser'', a short video game released to promote the cancelled video game ''Silent Hills'' * Porcupine Tree, a British progressive rock group ...
\sum_r w I_x -r_y -r & \sum_r w (I_y -r^2 \end Here the summation index ''r'' ranges over a finite set of index pairs (the "window", typically \\times\ for some ''m''), and ''w'' 'r''is a fixed "window weight" that depends on ''r'', such that the sum of all weights is 1. The values I_x I_y /math> are the partial derivatives sampled at pixel ''p''; which, for instance, may be estimated from by I by
finite difference A finite difference is a mathematical expression of the form . If a finite difference is divided by , one gets a difference quotient. The approximation of derivatives by finite differences plays a central role in finite difference methods for th ...
formulas. The formula of the structure tensor can be written also as S_w \sum_r w S_0 -r/math>, where S_0 is the matrix-valued array such that : S_0 = \begin (I_x ^2 & I_x _y \\
0pt PT, Pt, or pt may refer to: Arts and entertainment * ''P.T.'' (video game), acronym for ''Playable Teaser'', a short video game released to promote the cancelled video game ''Silent Hills'' * Porcupine Tree, a British progressive rock group ...
I_x _y & (I_y ^2 \end


Interpretation

The importance of the 2D structure tensor S_w stems from the fact
eigenvalue In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted ...
s \lambda_1,\lambda_2 (which can be ordered so that \lambda_1 \geq \lambda_2\geq 0) and the corresponding
eigenvector In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted b ...
s e_1,e_2 summarize the distribution of the
gradient In vector calculus, the gradient of a scalar-valued differentiable function of several variables is the vector field (or vector-valued function) \nabla f whose value at a point p is the "direction and rate of fastest increase". If the gradi ...
\nabla I = (I_x,I_y) of I within the window defined by w centered at p. Namely, if \lambda_1 > \lambda_2, then e_1 (or -e_1) is the direction that is maximally aligned with the gradient within the window. In particular, if \lambda_1 > 0, \lambda_2 = 0 then the gradient is always a multiple of e_1 (positive, negative or zero); this is the case if and only if I within the window varies along the direction e_1 but is constant along e_2. This condition of eigenvalues is also called linear symmetry condition because then the iso-curves of I consist in parallel lines, i.e there exists a one dimensional function g which can generate the two dimensional function I as I(x,y)=g(d^\text p) for some constant vector d=(d_x,d_y)^T and the coordinates p=(x,y)^T . If \lambda_1 = \lambda_2, on the other hand, the gradient in the window has no predominant direction; which happens, for instance, when the image has rotational symmetry within that window. This condition of eigenvalues is also called balanced body, or directional equilibrium condition because it holds when all gradient directions in the window are equally frequent/probable. Furthermore, the condition \lambda_1 = \lambda_2 = 0 happens if and only if the function I is constant (\nabla I = (0,0)) within W. More generally, the value of \lambda_k , for ''k''=1 or ''k''=2, is the w-weighted average, in the neighborhood of ''p'', of the square of the
directional derivative In mathematics, the directional derivative of a multivariable differentiable (scalar) function along a given vector v at a given point x intuitively represents the instantaneous rate of change of the function, moving through x with a velocity ...
of I along e_k. The relative discrepancy between the two eigenvalues of S_w is an indicator of the degree of
anisotropy Anisotropy () is the property of a material which allows it to change or assume different properties in different directions, as opposed to isotropy. It can be defined as a difference, when measured along different axes, in a material's physi ...
of the gradient in the window, namely how strongly is it biased towards a particular direction (and its opposite). This attribute can be quantified by the coherence, defined as :c_w=\left(\frac\right)^2 if \lambda_2>0. This quantity is 1 when the gradient is totally aligned, and 0 when it has no preferred direction. The formula is undefined, even in the limit, when the image is constant in the window (\lambda_1=\lambda_2=0). Some authors define it as 0 in that case. Note that the average of the gradient \nabla I inside the window is not a good indicator of anisotropy. Aligned but oppositely oriented gradient vectors would cancel out in this average, whereas in the structure tensor they are properly added together. This is a reason for why (\nabla I)(\nabla I)^\text is used in the averaging of the structure tensor to optimize the direction instead of \nabla I. By expanding the effective radius of the window function w (that is, increasing its variance), one can make the structure tensor more robust in the face of noise, at the cost of diminished spatial resolution.T. Lindeberg (1993),
Scale-Space Theory in Computer Vision
'. Kluwer Academic Publishers, (see sections 14.4.1 and 14.2.3 on pages 359–360 and 355–356 for detailed statements about how the multi-scale second-moment matrix/structure tensor defines a true and uniquely determined multi-scale representation of directional data).
The formal basis for this property is described in more detail below, where it is shown that a multi-scale formulation of the structure tensor, referred to as the multi-scale structure tensor, constitutes a ''true multi-scale representation of directional data under variations of the spatial extent of the window function''.


Complex version

The interpretation and implementation of the 2D structure tensor becomes particularly accessible using complex numbers. The structure tensor consists in 3 real numbers : S_w(p) = \begin \mu_ & \mu_ \\
0pt PT, Pt, or pt may refer to: Arts and entertainment * ''P.T.'' (video game), acronym for ''Playable Teaser'', a short video game released to promote the cancelled video game ''Silent Hills'' * Porcupine Tree, a British progressive rock group ...
\mu_ & \mu_ \end where \mu_ =\int (w(r) (I_x(p-r))^2\,d r , \mu_ =\int (w(r) (I_y(p-r))^2\,d r and \mu_ =\int w(r) I_x(p-r)I_y(p-r)\,d r in which integrals can be replaced by summations for discrete representation. Using
Parseval's identity In mathematical analysis, Parseval's identity, named after Marc-Antoine Parseval, is a fundamental result on the summability of the Fourier series of a function. Geometrically, it is a generalized Pythagorean theorem for inner-product spaces (which ...
it is clear that the three real numbers are the second order moments of the power spectrum of I. The following second order complex moment of the power spectrum of I can then be written as : \kappa_ =\mu_-\mu_+i2\mu_=\int (w(r) (I_x(p-r)+i I_y(p-r))^2\,d r =(\lambda_1-\lambda_2)\exp(i2\phi) where i=\sqrt and \phi is the direction angle of the most significant eigenvector of the structure tensor \phi=\angle whereas \lambda_1 and \lambda_2 are the most and the least significant eigenvalues. From, this it follows that \kappa_ contains both a certainty , \kappa_, =\lambda_1-\lambda_2 and the optimal direction in double angle representation since it is a complex number consisting of two real numbers. It follows also that if the gradient is represented as a complex number, and is remapped by squaring (i.e. the argument angles of the complex gradient is doubled), then averaging acts as an optimizer in the mapped domain, since it directly delivers both the optimal direction (in double angle representation) and the associated certainty. The complex number represents thus how much linear structure (linear symmetry) there is in image I, and the complex number is obtained directly by averaging the gradient in its (complex) double angle representation without computing the eigenvalues and the eigenvectors explicitly. Likewise the following second order complex moment of the power spectrum of I, which happens to be always real because I is real, : \kappa_ =\mu_+\mu_=\int (w(r) , I_x(p-r)+i I_y(p-r), ^2\,d r =\lambda_1+\lambda_2 can be obtained, with \lambda_1 and \lambda_2 being the eigenvalues as before. Notice that this time the magnitude of the complex gradient is squared (which is always real). However, decomposing the structure tensor in its eigenvectors yields its tensor components as : S_w(p) = \lambda_1 e_1e_1^\text+ \lambda_2 e_2e_2^\text =(\lambda_1 -\lambda_2)e_1e_1^\text+ \lambda_2( e_1e_1^\text+e_2e_2^\text)= (\lambda_1 -\lambda_2)e_1e_1^\text+ \lambda_2 E where E is the identity matrix in 2D because the two eigenvectors are always orthogonal (and sum to unity). The first term in the last expression of the decomposition, (\lambda_1 -\lambda_2)e_1e_1^\text, represents the linear symmetry component of the structure tensor containing all directional information (as a rank-1 matrix), whereas the second term represents the balanced body component of the tensor, which lacks any directional information (containing an identity matrix E). To know how much directional information there is in I is then the same as checking how large \lambda_1-\lambda_2 is compared to \lambda_2. Evidently, \kappa_ is the complex equivalent of the first term in the tensor decomposition, whereas (, \kappa_, -\kappa_)/2=\lambda_2is the equivalent of the second term. Thus the two scalars, comprising three real numbers, : \begin \kappa_ =(\lambda_1-\lambda_2)\exp(i2\phi)&=&w*(h*I)^2\\ \kappa_ =\lambda_1+\lambda_2&=&w*, h*I, ^2\\ \end where h(x,y)=(x+iy)\exp(-(x^2+y^2)/(2\sigma^2)) is the (complex) gradient filter, and * is convolution, constitute a complex representation of the 2D Structure Tensor. As discussed here and elsewhere w defines the local image which is usually a Gaussian (with a certain variance defining the outer scale), and \sigma is the (inner scale) parameter determining the effective frequency range in which the orientation 2\phi is to be estimated. The elegance of the complex representation stems from that the two components of the structure tensor can be obtained as averages and independently. In turn, this means that \kappa_ and \kappa_ can be used in a scale space representation to describe the evidence for presence of unique orientation and the evidence for the alternative hypothesis, the presence of multiple balanced orientations, without computing the eigenvectors and eigenvalues. A functional, such as squaring the complex numbers have to this date not been shown to exist for structure tensors with dimensions higher than two. In Bigun 91, it has been put forward with due argument that this is because complex numbers are commutative algebras whereas quaternions, the possible candidate to construct such a functional by, constitute a non-commutative algebra. The complex representation of the structure tensor is frequently used in fingerprint analysis to obtain direction maps containing certainties which in turn are used to enhance them, to find the locations of the global (cores and deltas) and local (minutia) singularities, as well as automatically evaluate the quality of the fingerprints.


The 3D structure tensor


Definition

The structure tensor can be defined also for a function I of three variables ''p''=(''x'',''y'',''z'') in an entirely analogous way. Namely, in the continuous version we have S_w(p) = \int w(r) S_0(p-r)\,d r, where : S_0(p) = \begin (I_x(p))^2 & I_x(p)I_y(p) & I_x(p)I_z(p) \\
0pt PT, Pt, or pt may refer to: Arts and entertainment * ''P.T.'' (video game), acronym for ''Playable Teaser'', a short video game released to promote the cancelled video game ''Silent Hills'' * Porcupine Tree, a British progressive rock group ...
I_x(p)I_y(p) & (I_y(p))^2 & I_y(p)I_z(p) \\
0pt PT, Pt, or pt may refer to: Arts and entertainment * ''P.T.'' (video game), acronym for ''Playable Teaser'', a short video game released to promote the cancelled video game ''Silent Hills'' * Porcupine Tree, a British progressive rock group ...
I_x(p)I_z(p) & I_y(p)I_z(p) & (I_z(p))^2 \end where I_x,I_y,I_z are the three partial derivatives of I, and the integral ranges over \mathbb^3. In the discrete version,S_w \sum_r w S_0 -r/math>, where : S_0 = \begin (I_x ^2 & I_x _y & I_x _z \\
0pt PT, Pt, or pt may refer to: Arts and entertainment * ''P.T.'' (video game), acronym for ''Playable Teaser'', a short video game released to promote the cancelled video game ''Silent Hills'' * Porcupine Tree, a British progressive rock group ...
I_x _y & (I_y ^2 & I_y _z \
0pt PT, Pt, or pt may refer to: Arts and entertainment * ''P.T.'' (video game), acronym for ''Playable Teaser'', a short video game released to promote the cancelled video game ''Silent Hills'' * Porcupine Tree, a British progressive rock group ...
I_x _z & I_y _z & (I_z ^2 \end and the sum ranges over a finite set of 3D indices, usually \\times\\times\ for some ''m''.


Interpretation

As in the two-dimensional case, the eigenvalues \lambda_1,\lambda_2,\lambda_3 of S_w /math>, and the corresponding eigenvectors \hat_1,\hat_2,\hat_3, summarize the distribution of gradient directions within the neighborhood of ''p'' defined by the window w. This information can be visualized as an
ellipsoid An ellipsoid is a surface that may be obtained from a sphere by deforming it by means of directional scalings, or more generally, of an affine transformation. An ellipsoid is a quadric surface;  that is, a surface that may be defined as the z ...
whose semi-axes are equal to the eigenvalues and directed along their corresponding eigenvectors. In particular, if the ellipsoid is stretched along one axis only, like a cigar (that is, if \lambda_1 is much larger than both \lambda_2 and \lambda_3), it means that the gradient in the window is predominantly aligned with the direction e_1, so that the
isosurface An isosurface is a three-dimensional analog of an isoline. It is a surface that represents points of a constant value (e.g. pressure, temperature, velocity, density) within a volume of space; in other words, it is a level set of a continuous f ...
s of I tend to be flat and perpendicular to that vector. This situation occurs, for instance, when ''p'' lies on a thin plate-like feature, or on the smooth boundary between two regions with contrasting values. If the ellipsoid is flattened in one direction only, like a pancake (that is, if \lambda_3 is much smaller than both \lambda_1 and \lambda_2), it means that the gradient directions are spread out but perpendicular to e_3; so that the isosurfaces tend to be like tubes parallel to that vector. This situation occurs, for instance, when ''p'' lies on a thin line-like feature, or on a sharp corner of the boundary between two regions with contrasting values. Finally, if the ellipsoid is roughly spherical (that is, if \lambda_1\approx\lambda_2\approx\lambda_3), it means that the gradient directions in the window are more or less evenly distributed, with no marked preference; so that the function I is mostly isotropic in that neighborhood. This happens, for instance, when the function has
spherical symmetry In geometry, circular symmetry is a type of continuous symmetry for a planar object that can be rotated by any arbitrary angle and map onto itself. Rotational circular symmetry is isomorphic with the circle group in the complex plane, or the ...
in the neighborhood of ''p''. In particular, if the ellipsoid degenerates to a point (that is, if the three eigenvalues are zero), it means that I is constant (has zero gradient) within the window.


The multi-scale structure tensor

The structure tensor is an important tool in
scale space Scale-space theory is a framework for multi-scale signal representation developed by the computer vision, image processing and signal processing communities with complementary motivations from physics and biological vision. It is a formal theory ...
analysis. The multi-scale structure tensor (or multi-scale second moment matrix) of a function I is in contrast to other one-parameter scale-space features an image descriptor that is defined over ''two'' scale parameters. One scale parameter, referred to as ''local scale'' t, is needed for determining the amount of pre-smoothing when computing the image gradient (\nabla I)(x; t). Another scale parameter, referred to as ''integration scale'' s, is needed for specifying the spatial extent of the window function w(\xi; s) that determines the weights for the region in space over which the components of the outer product of the gradient by itself (\nabla I)(\nabla I)^\text are accumulated. More precisely, suppose that I is a real-valued signal defined over \mathbb^k. For any local scale t > 0, let a multi-scale representation I(x; t) of this signal be given by I(x; t) = h(x; t)*I(x) where h(x; t) represents a pre-smoothing kernel. Furthermore, let (\nabla I)(x; t) denote the gradient of the scale space representation. Then, the ''multi-scale structure tensor/second-moment matrix'' is defined by J. Garding and T. Lindeberg (1996).
"Direct computation of shape cues using scale-adapted spatial derivative operators
', International Journal of Computer Vision, volume 17, issue 2, pages 163–191.
: \mu(x; t, s) = \int_ (\nabla I)(x-\xi; t) \, (\nabla I)^\text(x-\xi; t) \, w(\xi; s) \, d\xi Conceptually, one may ask if it would be sufficient to use any self-similar families of smoothing functions h(x; t) and w(\xi; s). If one naively would apply, for example, a box filter, however, then non-desirable artifacts could easily occur. If one wants the multi-scale structure tensor to be well-behaved over both increasing local scales t and increasing integration scales s, then it can be shown that both the smoothing function and the window function ''have to'' be Gaussian. The conditions that specify this uniqueness are similar to the scale-space axioms that are used for deriving the uniqueness of the Gaussian kernel for a regular Gaussian
scale space Scale-space theory is a framework for multi-scale signal representation developed by the computer vision, image processing and signal processing communities with complementary motivations from physics and biological vision. It is a formal theory ...
of image intensities. There are different ways of handling the two-parameter scale variations in this family of image descriptors. If we keep the local scale parameter t fixed and apply increasingly broadened versions of the window function by increasing the integration scale parameter s only, then we obtain a ''true formal scale space representation of the directional data computed at the given local scale'' t. If we couple the local scale and integration scale by a ''relative integration scale'' r \geq 1, such that s = r t then for any fixed value of r, we obtain a reduced self-similar one-parameter variation, which is frequently used to simplify computational algorithms, for example in
corner detection Corner detection is an approach used within computer vision systems to extract certain kinds of features and infer the contents of an image. Corner detection is frequently used in motion detection, image registration, video tracking, image mos ...
, interest point detection, texture analysis and image matching. By varying the relative integration scale r \geq 1 in such a self-similar scale variation, we obtain another alternative way of parameterizing the multi-scale nature of directional data obtained by increasing the integration scale. A conceptually similar construction can be performed for discrete signals, with the convolution integral replaced by a convolution sum and with the continuous Gaussian kernel g(x; t) replaced by the discrete Gaussian kernel T(n; t): : \mu(x; t, s) = \sum_ (\nabla I)(x-n; t) \, (\nabla I)^\text(x-n; t) \, w(n; s) When quantizing the scale parameters t and s in an actual implementation, a finite geometric progression \alpha^i is usually used, with ''i'' ranging from 0 to some maximum scale index ''m''. Thus, the discrete scale levels will bear certain similarities to image pyramid, although spatial subsampling may not necessarily be used in order to preserve more accurate data for subsequent processing stages.


Applications

The eigenvalues of the structure tensor play a significant role in many image processing algorithms, for problems like
corner detection Corner detection is an approach used within computer vision systems to extract certain kinds of features and infer the contents of an image. Corner detection is frequently used in motion detection, image registration, video tracking, image mos ...
, interest point detection, and feature tracking. The structure tensor also plays a central role in the Lucas-Kanade optical flow algorithm, and in its extensions to estimate
affine shape adaptation Affine shape adaptation is a methodology for iteratively adapting the shape of the smoothing kernels in an affine group of smoothing kernels to the local image structure in neighbourhood region of a specific image point. Equivalently, affine shape ...
; where the magnitude of \lambda_2 is an indicator of the reliability of the computed result. The tensor has been used for
scale space Scale-space theory is a framework for multi-scale signal representation developed by the computer vision, image processing and signal processing communities with complementary motivations from physics and biological vision. It is a formal theory ...
analysis, estimation of local surface orientation from monocular or binocular cues, non-linear fingerprint enhancement, diffusion-based image processing, and several other image processing problems. The structure tensor can be also applied in
geology Geology () is a branch of natural science concerned with Earth and other astronomical objects, the features or rocks of which it is composed, and the processes by which they change over time. Modern geology significantly overlaps all other Ear ...
to filter
seismic Seismology (; from Ancient Greek σεισμός (''seismós'') meaning "earthquake" and -λογία (''-logía'') meaning "study of") is the scientific study of earthquakes and the propagation of elastic waves through the Earth or through other ...
data.


Processing spatio-temporal video data with the structure tensor

The three-dimensional structure tensor has been used to analyze three-dimensional video data (viewed as a function of ''x'', ''y'', and time ''t''). If one in this context aims at image descriptors that are ''invariant'' under
Galilean transformation In physics, a Galilean transformation is used to transform between the coordinates of two reference frames which differ only by constant relative motion within the constructs of Newtonian physics. These transformations together with spatial rotatio ...
s, to make it possible to compare image measurements that have been obtained under variations of a priori unknown image velocities v = (v_x, v_y)^\text : \begin x' \\ y' \\ t' \end = G \begin x \\ y \\ t \end = \begin x - v_x \, t \\ y - v_y \, t \\ t \end , it is, however, from a computational viewpoint preferable to parameterize the components in the structure tensor/second-moment matrix S using the notion of ''Galilean diagonalization'' : S' = R_\text^ \, G^ \, S \, G^ \, R_\text^ = \begin \nu_1 & \, & \, \\ \, & \nu_2 & \, \\ \, & \, & \nu_3 \end where G denotes a Galilean transformation of spacetime and R_\text a two-dimensional rotation over the spatial domain, compared to the abovementioned use of eigenvalues of a 3-D structure tensor, which corresponds to an eigenvalue decomposition and a (non-physical) three-dimensional rotation of spacetime : S'' = R_\text^ \, S \, R_\text^ = \begin \lambda_1 & & \\ & \lambda_2 & \\ & & \lambda_3 \end . To obtain true Galilean invariance, however, also the shape of the spatio-temporal window function needs to be adapted, corresponding to the transfer of
affine shape adaptation Affine shape adaptation is a methodology for iteratively adapting the shape of the smoothing kernels in an affine group of smoothing kernels to the local image structure in neighbourhood region of a specific image point. Equivalently, affine shape ...
from spatial to spatio-temporal image data. In combination with local spatio-temporal histogram descriptors, these concepts together allow for Galilean invariant recognition of spatio-temporal events.


See also

*
Tensor In mathematics, a tensor is an algebraic object that describes a multilinear relationship between sets of algebraic objects related to a vector space. Tensors may map between different objects such as vectors, scalars, and even other tenso ...
*
Tensor operator In pure and applied mathematics, quantum mechanics and computer graphics, a tensor operator generalizes the notion of operators which are scalars and vectors. A special class of these are spherical tensor operators which apply the notion of th ...
*
Directional derivative In mathematics, the directional derivative of a multivariable differentiable (scalar) function along a given vector v at a given point x intuitively represents the instantaneous rate of change of the function, moving through x with a velocity ...
*
Gaussian Carl Friedrich Gauss (1777–1855) is the eponym of all of the topics listed below. There are over 100 topics all named after this German mathematician and scientist, all in the fields of mathematics, physics, and astronomy. The English eponymo ...
*
Corner detection Corner detection is an approach used within computer vision systems to extract certain kinds of features and infer the contents of an image. Corner detection is frequently used in motion detection, image registration, video tracking, image mos ...
*
Edge detection Edge detection includes a variety of mathematical methods that aim at identifying edges, curves in a digital image at which the image brightness changes sharply or, more formally, has discontinuities. The same problem of finding discontinuitie ...
* Lucas-Kanade method *
Affine shape adaptation Affine shape adaptation is a methodology for iteratively adapting the shape of the smoothing kernels in an affine group of smoothing kernels to the local image structure in neighbourhood region of a specific image point. Equivalently, affine shape ...
* Generalized structure tensor


References


Resources


Download MATLAB SourceStructure Tensor Tutorial (Original)
{{DEFAULTSORT:Structure Tensor Tensors Feature detection (computer vision)