The Mahalanobis distance is a measure of the distance between a point ''P'' and a
distribution Distribution may refer to:
Mathematics
* Distribution (mathematics), generalized functions used to formulate solutions of partial differential equations
*Probability distribution, the probability of a particular value or value range of a vari ...
''D'', introduced by
P. C. Mahalanobis in 1936. Mahalanobis's definition was prompted by the problem of identifying the similarities of skulls based on measurements in 1927.
It is a multi-dimensional generalization of the idea of measuring how many
standard deviations away ''P'' is from the
mean
There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value ( magnitude and sign) of a given data set.
For a data set, the '' ar ...
of ''D''. This distance is zero for ''P'' at the mean of ''D'' and grows as ''P'' moves away from the mean along each
principal component axis. If each of these axes is re-scaled to have unit variance, then the Mahalanobis distance corresponds to standard
Euclidean distance
In mathematics, the Euclidean distance between two points in Euclidean space is the length of a line segment between the two points.
It can be calculated from the Cartesian coordinates of the points using the Pythagorean theorem, therefore ...
in the transformed space. The Mahalanobis distance is thus
unitless
A dimensionless quantity (also known as a bare quantity, pure quantity, or scalar quantity as well as quantity of dimension one) is a quantity to which no physical dimension is assigned, with a corresponding SI unit of measurement of one (or 1) ...
,
scale-invariant, and takes into account the
correlations of the
data set.
Definition
Given a probability distribution
on
, with mean
and positive-definite covariance matrix
, the Mahalanobis distance of a point
from
is
Given two points
and
in
, the Mahalanobis distance between them with respect to
is
which means that
.
Since
is
positive-definite, so is
, thus the square roots are always defined.
We can find useful decompositions of the squared Mahalanobis distance that help to explain some reasons for the outlyingness of multivariate observations and also provide a graphical tool for identifying outliers.
By the
spectral theorem
In mathematics, particularly linear algebra and functional analysis, a spectral theorem is a result about when a linear operator or matrix can be diagonalized (that is, represented as a diagonal matrix in some basis). This is extremely useful be ...
,
can be decomposed as
for some real
matrix, which gives us the equivalent definition
where
is the Euclidean norm. That is, the Mahalanobis distance is the Euclidean distance after a
whitening transformation A whitening transformation or sphering transformation is a linear transformation that transforms a vector of random variables with a known covariance matrix into a set of new variables whose covariance is the identity matrix, meaning that they are u ...
.
The existence of
is guaranteed by the spectral theorem, but it is not unique. Different choices have different theoretical and practical advantages.
In practice, the distribution
is usually the
sample distribution from a set of
IID samples from an underlying unknown distribution, so
is the sample mean, and
is the covariance matrix of the samples.
When the
affine span
In mathematics, the affine hull or affine span of a set ''S'' in Euclidean space R''n'' is the smallest affine set containing ''S'', or equivalently, the intersection of all affine sets containing ''S''. Here, an ''affine set'' may be defined as ...
of the samples is not the entire
, the covariance matrix would not be positive-definite, which means the above definition would not work. This is repaired by noting that, in general, the Mahalanobis distance is preserved under any full-rank affine transformation of the affine span of the samples. So in case the affine span is not the entire
, the samples can be first projected non-degenerately to
, where
is the dimension of the affine span of the samples, then the Mahalanobis distance can be computed as usual.
Intuitive explanation
Consider the problem of estimating the probability that a test point in ''N''-dimensional
Euclidean space
Euclidean space is the fundamental space of geometry, intended to represent physical space. Originally, that is, in Euclid's ''Elements'', it was the three-dimensional space of Euclidean geometry, but in modern mathematics there are Euclidea ...
belongs to a set, where we are given sample points that definitely belong to that set. Our first step would be to find the
centroid or center of mass of the sample points. Intuitively, the closer the point in question is to this center of mass, the more likely it is to belong to the set.
However, we also need to know if the set is spread out over a large range or a small range, so that we can decide whether a given distance from the center is noteworthy or not. The simplistic approach is to estimate the
standard deviation
In statistics, the standard deviation is a measure of the amount of variation or dispersion of a set of values. A low standard deviation indicates that the values tend to be close to the mean (also called the expected value) of the set, whil ...
of the distances of the sample points from the center of mass. If the distance between the test point and the center of mass is less than one standard deviation, then we might conclude that it is highly probable that the test point belongs to the set. The further away it is, the more likely that the test point should not be classified as belonging to the set.
This intuitive approach can be made quantitative by defining the normalized distance between the test point and the set to be
, which reads:
. By plugging this into the normal distribution we can derive the probability of the test point belonging to the set.
The drawback of the above approach was that we assumed that the sample points are distributed about the center of mass in a spherical manner. Were the distribution to be decidedly non-spherical, for instance ellipsoidal, then we would expect the probability of the test point belonging to the set to depend not only on the distance from the center of mass, but also on the direction. In those directions where the ellipsoid has a short axis the test point must be closer, while in those where the axis is long the test point can be further away from the center.
Putting this on a mathematical basis, the ellipsoid that best represents the set's probability distribution can be estimated by building the covariance matrix of the samples. The Mahalanobis distance is the distance of the test point from the center of mass divided by the width of the ellipsoid in the direction of the test point.
Normal distributions
For a
normal distribution
In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is
:
f(x) = \frac e^
The parameter \mu ...
in any number of dimensions, the probability density of an observation
is uniquely determined by the Mahalanobis distance
:
:
Specifically,
follows the
chi-squared distribution
In probability theory and statistics, the chi-squared distribution (also chi-square or \chi^2-distribution) with k degrees of freedom is the distribution of a sum of the squares of k independent standard normal random variables. The chi-squar ...
with
degrees of freedom, where
is the number of dimensions of the normal distribution. If the number of dimensions is 2, for example, the probability of a particular calculated
being less than some threshold
is
. To determine a threshold to achieve a particular probability,
, use
, for 2 dimensions. For number of dimensions other than 2, the cumulative chi-squared distribution should be consulted.
In a normal distribution, the region where the Mahalanobis distance is less than one (i.e. the region inside the ellipsoid at distance one) is exactly the region where the probability distribution is
concave
Concave or concavity may refer to:
Science and technology
* Concave lens
* Concave mirror
Mathematics
* Concave function, the negative of a convex function
* Concave polygon, a polygon which is not convex
* Concave set
In geometry, a subset o ...
.
Mahalanobis distance is proportional, for a normal distribution, to the square root of the negative
log-likelihood (after adding a constant so the minimum is at zero).
Other forms of multivariate location and scatter
The sample mean and covariance matrix can be quite sensitive to outliers, therefore other approaches to calculating the multivariate location and scatter of data are also commonly used when calculating the Mahalanobis distance. The Minimum Covariance Determinant approach estimates multivariate location and scatter from a subset numbering
data points that has the smallest variance-covariance matrix determinant. The Minimum Volume Ellipsoid approach is similar to the Minimum Covariance Determinant approach in that it works with a subset of size
data points, but the Minimum Volume Ellipsoid estimates multivariate location and scatter from the ellipsoid of minimal volume that encapsulates the
data points. Each method varies in its definition of the distribution of the data, and therefore produces different Mahalanobis distances. The Minimum Covariance Determinant and Minimum Volume Ellipsoid approaches are more robust to samples that contain outliers, while the sample mean and covariance matrix tends to be more reliable with small and biased data sets.
Relationship to normal random variables
In general, given a normal (
Gaussian
Carl Friedrich Gauss (1777–1855) is the eponym of all of the topics listed below.
There are over 100 topics all named after this German mathematician and scientist, all in the fields of mathematics, physics, and astronomy. The English eponym ...
) random variable
with variance
and mean
, any other normal random variable
(with mean
and variance
) can be defined in terms of
by the equation
Conversely, to recover a normalized random variable from any normal random variable, one can typically solve for
. If we square both sides, and take the square-root, we will get an equation for a metric that looks a lot like the Mahalanobis distance:
The resulting magnitude is always non-negative and varies with the distance of the data from the mean, attributes that are convenient when trying to define a model for the data.
Relationship to leverage
Mahalanobis distance is closely related to the
leverage statistic,
, but has a different scale:
Applications
Mahalanobis distance is widely used in
cluster analysis
Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense) to each other than to those in other groups (clusters). It is a main task of ...
and
classification techniques. It is closely related to
Hotelling's T-square distribution used for multivariate statistical testing and Fisher's
Linear Discriminant Analysis
Linear discriminant analysis (LDA), normal discriminant analysis (NDA), or discriminant function analysis is a generalization of Fisher's linear discriminant, a method used in statistics and other fields, to find a linear combination of features ...
that is used for
supervised classification.
In order to use the Mahalanobis distance to classify a test point as belonging to one of ''N'' classes, one first
estimates the covariance matrix of each class, usually based on samples known to belong to each class. Then, given a test sample, one computes the Mahalanobis distance to each class, and classifies the test point as belonging to that class for which the Mahalanobis distance is minimal.
Mahalanobis distance and leverage are often used to detect
outliers, especially in the development of
linear regression
In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is cal ...
models. A point that has a greater Mahalanobis distance from the rest of the sample population of points is said to have higher leverage since it has a greater influence on the slope or coefficients of the regression equation. Mahalanobis distance is also used to determine multivariate outliers. Regression techniques can be used to determine if a specific case within a sample population is an outlier via the combination of two or more variable scores. Even for normal distributions, a point can be a multivariate outlier even if it is not a univariate outlier for any variable (consider a probability density concentrated along the line
, for example), making Mahalanobis distance a more sensitive measure than checking dimensions individually.
Mahalanobis distance has also been used in ecological niche modelling, as the convex elliptical shape of the distances relates well to the concept of the
fundamental niche
In ecology, a niche is the match of a species to a specific environmental condition.
Three variants of ecological niche are described by
It describes how an organism or population responds to the distribution of resources and competitors (for ...
.
Another example of usage is in finance, where Mahalanobis distance has been used to compute an indicator called the "turbulence index", which is a statistical measure of financial markets abnormal behaviour. An implementation as a Web API of this indicator is available online.
Software implementations
Many programs and statistics packages, such as
R,
Python, etc., include implementations of Mahalanobis distance.
See also
*
Bregman divergence (the Mahalanobis distance is an example of a Bregman divergence)
*
Bhattacharyya distance related, for measuring similarity between data sets (and not between a point and a data set)
*
Hamming distance identifies the difference bit by bit of two strings
*
Hellinger distance, also a measure of distance between data sets
*
Similarity learning
Similarity learning is an area of supervised machine learning in artificial intelligence. It is closely related to regression and classification, but the goal is to learn a similarity function that measures how similar or related two objects are. ...
, for other approaches to learn a distance metric from examples.
References
External links
*
Mahalanobis distance tutorial– interactive online program and spreadsheet computation
– overview of Mahalanobis distance, including MATLAB code
What is Mahalanobis distance?– intuitive, illustrated explanation, from Rick Wicklin on blogs.sas.com
{{DEFAULTSORT:Mahalanobis Distance
Statistical distance
Multivariate statistics
Distance