HOME

TheInfoList



OR:

In
mathematics Mathematics is a field of study that discovers and organizes methods, Mathematical theory, theories and theorems that are developed and Mathematical proof, proved for the needs of empirical sciences and mathematics itself. There are many ar ...
, the
Wasserstein Wasserstein is a surname and can refer to: * Abraham Wasserstein (1921–1995), a German-born British and Israeli classicist * Bernard Wasserstein (born 1948), a British historian * Bruce Wasserstein (1947–2009), an American former investment b ...
distance or
Kantorovich Leonid Vitalyevich Kantorovich (, ; 19 January 19127 April 1986) was a Soviet mathematician and economist, known for his theory and development of techniques for the optimal allocation of resources. He is regarded as the founder of linear programm ...
Rubinstein Rubinstein is a surname of Ashkenazi Jews. It comes from German and Yiddish, where it means "ruby-stone". Notable persons named Rubinstein include: A–E * Akiba Rubinstein (1880–1961), Polish chess grandmaster * Amnon Rubinstein (1931-2024), I ...
metric is a
distance function In mathematics, a metric space is a set together with a notion of ''distance'' between its elements, usually called points. The distance is measured by a function called a metric or distance function. Metric spaces are a general setting fo ...
defined between
probability distributions In probability theory and statistics, a probability distribution is a function that gives the probabilities of occurrence of possible events for an experiment. It is a mathematical description of a random phenomenon in terms of its sample spac ...
on a given
metric space In mathematics, a metric space is a Set (mathematics), set together with a notion of ''distance'' between its Element (mathematics), elements, usually called point (geometry), points. The distance is measured by a function (mathematics), functi ...
M. It is named after
Leonid Vaseršteĭn Leonid Nisonovich Vaserstein () is a Russian- American mathematician, currently Professor of Mathematics at Penn State University. His research is focused on algebra and dynamical systems. He is well known for providing a simple proof of the Quil ...
. Intuitively, if each distribution is viewed as a unit amount of earth (soil) piled on ''M'', the metric is the minimum "cost" of turning one pile into the other, which is assumed to be the amount of earth that needs to be moved times the mean distance it has to be moved. This problem was first formalised by
Gaspard Monge Gaspard Monge, Comte de Péluse (; 9 May 1746 – 28 July 1818) was a French mathematician, commonly presented as the inventor of descriptive geometry, (the mathematical basis of) technical drawing, and the father of differential geometry. Dur ...
in 1781. Because of this analogy, the metric is known in
computer science Computer science is the study of computation, information, and automation. Computer science spans Theoretical computer science, theoretical disciplines (such as algorithms, theory of computation, and information theory) to Applied science, ...
as the
earth mover's distance In computer science, the earth mover's distance (EMD) is a measure of dissimilarity between two frequency distributions, densities, or measures, over a metric space ''D''. Informally, if the distributions are interpreted as two different ways of ...
. The name "Wasserstein distance" was coined by R. L. Dobrushin in 1970, after learning of it in the work of
Leonid Vaseršteĭn Leonid Nisonovich Vaserstein () is a Russian- American mathematician, currently Professor of Mathematics at Penn State University. His research is focused on algebra and dynamical systems. He is well known for providing a simple proof of the Quil ...
on Markov processes describing large systems of automata (Russian, 1969). However the metric was first defined by
Leonid Kantorovich Leonid Vitalyevich Kantorovich (, ; 19 January 19127 April 1986) was a Soviet mathematician and economist, known for his theory and development of techniques for the optimal allocation of resources. He is regarded as the founder of linear programm ...
in ''The Mathematical Method of Production Planning and Organization'' (Russian original 1939) in the context of optimal transport planning of goods and materials. Some scholars thus encourage use of the terms "Kantorovich metric" and "Kantorovich distance". Most English-language publications use the
German German(s) may refer to: * Germany, the country of the Germans and German things **Germania (Roman era) * Germans, citizens of Germany, people of German ancestry, or native speakers of the German language ** For citizenship in Germany, see also Ge ...
spelling "Wasserstein" (attributed to the name "Vaseršteĭn" () being of
Yiddish Yiddish, historically Judeo-German, is a West Germanic language historically spoken by Ashkenazi Jews. It originated in 9th-century Central Europe, and provided the nascent Ashkenazi community with a vernacular based on High German fused with ...
origin).


Definition

Let (M,d) be a
metric space In mathematics, a metric space is a Set (mathematics), set together with a notion of ''distance'' between its Element (mathematics), elements, usually called point (geometry), points. The distance is measured by a function (mathematics), functi ...
that is a
Polish space In the mathematical discipline of general topology, a Polish space is a separable space, separable Completely metrizable space, completely metrizable topological space; that is, a space homeomorphic to a Complete space, complete metric space that h ...
. For p \in , +\infty/math>, the Wasserstein p-distance between two
probability measure In mathematics, a probability measure is a real-valued function defined on a set of events in a σ-algebra that satisfies Measure (mathematics), measure properties such as ''countable additivity''. The difference between a probability measure an ...
s \mu and \nu on M with finite p- moments is :W_p(\mu, \nu) = \inf_ \left(\mathbf_ d(x, y)^p \right)^\frac, where \Gamma(\mu, \nu) is the set of all couplings of \mu and \nu; W_\infty(\mu, \nu) is defined to be :\lim_ W_p(\mu, \nu) and corresponds to a
supremum norm In mathematical analysis, the uniform norm (or ) assigns, to real- or complex-valued bounded functions defined on a set , the non-negative number :\, f\, _\infty = \, f\, _ = \sup\left\. This norm is also called the , the , the , or, when t ...
. Here, a coupling \gamma is a
joint probability A joint or articulation (or articular surface) is the connection made between bones, ossicles, or other hard structures in the body which link an animal's skeletal system into a functional whole.Saladin, Ken. Anatomy & Physiology. 7th ed. McGra ...
measure on M \times M whose marginals are \mu and \nu on the first and second factors, respectively. This means that for all measurable A\subset M, it fulfills \gamma(A\times M) = \mu(A) and \gamma(M\times A) = \nu(A).


Intuition and connection to optimal transport

One way to understand the above definition is to consider the optimal transport problem. That is, for a distribution of mass \mu(x) on a space X, we wish to transport the mass in such a way that it is transformed into the distribution \nu(x) on the same space; transforming the 'pile of earth' \mu to the pile \nu. This problem only makes sense if the pile to be created has the same mass as the pile to be moved; therefore without loss of generality assume that \mu and \nu are probability distributions containing a total mass of 1. Assume also that there is given some cost function c(x,y) \geq 0 that gives the cost of transporting a unit mass from the point x to the point y. A transport plan to move \mu into \nu can be described by a function \gamma(x,y) which gives the amount of mass to move from x to y. You can imagine the task as the need to move a pile of earth of shape \mu to the hole in the ground of shape \nu such that at the end, both the pile of earth and the hole in the ground completely vanish. In order for this plan to be meaningful, it must satisfy the following properties:
  1. the amount of earth moved out of point x must equal the amount that was there to begin with; that is, \int \gamma(x,y) \,\mathrm y = \mu(x), and
  2. the amount of earth moved into point y must equal the depth of the hole that was there at the beginning; that is, \int \gamma(x,y) \,\mathrm x = \nu(y).
That is, that the total mass moved ''out of'' an infinitesimal region around x must be equal to \mu(x) \mathrmx and the total mass moved ''into'' a region around y must be \nu(y)\mathrmy. This is equivalent to the requirement that \gamma be a
joint probability distribution A joint or articulation (or articular surface) is the connection made between bones, ossicles, or other hard structures in the body which link an animal's skeletal system into a functional whole.Saladin, Ken. Anatomy & Physiology. 7th ed. McGraw- ...
with marginals \mu and \nu. Thus, the infinitesimal mass transported from x to y is \gamma(x,y) \, \mathrm x \, \mathrm y, and the cost of moving is c(x,y) \gamma(x,y) \, \mathrm x \, \mathrm y, following the definition of the cost function. Therefore, the total cost of a transport plan \gamma is \iint c(x,y) \gamma(x,y) \, \mathrm x \, \mathrm y = \int c(x,y) \, \mathrm \gamma(x,y). The plan \gamma is not unique; the optimal transport plan is the plan with the minimal cost out of all possible transport plans. As mentioned, the requirement for a plan to be valid is that it is a joint distribution with marginals \mu and \nu; letting \Gamma denote the set of all such measures as in the first section, the cost of the optimal plan is C = \inf_ \int c(x,y) \, \mathrm \gamma(x,y). If the cost of a move is simply the distance between the two points, then the optimal cost is identical to the definition of the W_1 distance.


Examples


Point masses


Deterministic distributions

Let \mu_ = \delta_ and \mu_ = \delta_ be two
degenerate distribution In probability theory, a degenerate distribution on a measure space (E, \mathcal, \mu) is a probability distribution whose support is a null set with respect to \mu. For instance, in the -dimensional space endowed with the Lebesgue measure, an ...
s (i.e.
Dirac delta distribution In mathematical analysis, the Dirac delta function (or distribution), also known as the unit impulse, is a generalized function on the real numbers, whose value is zero everywhere except at zero, and whose integral over the entire real line ...
s) located at points a_ and a_ in \mathbb. There is only one possible coupling of these two measures, namely the point mass \delta_ located at (a_, a_) \in \mathbb^. Thus, using the usual
absolute value In mathematics, the absolute value or modulus of a real number x, is the non-negative value without regard to its sign. Namely, , x, =x if x is a positive number, and , x, =-x if x is negative (in which case negating x makes -x positive), ...
function as the distance function on \mathbb, for any p \geq 1, the p-Wasserstein distance between \mu_ and \mu_2 is W_p (\mu_1, \mu_2) = , a_1 - a_2 , . By similar reasoning, if \mu_ = \delta_ and \mu_ = \delta_ are point masses located at points a_ and a_ in \mathbb^, and we use the usual
Euclidean norm Euclidean space is the fundamental space of geometry, intended to represent physical space. Originally, in Euclid's ''Elements'', it was the three-dimensional space of Euclidean geometry, but in modern mathematics there are ''Euclidean spaces'' ...
on \mathbb^ as the distance function, then W_p(\mu_1, \mu_2) = \, a_1 - a_2 \, _2 .


Empirical distributions


=One dimension

= If P is an
empirical measure In probability theory, an empirical measure is a random measure arising from a particular realization of a (usually finite) sequence of random variables. The precise definition is found below. Empirical measures are relevant to mathematical sta ...
with samples X_1, \ldots, X_n and Q is an empirical measure with samples Y_1, \ldots, Y_n, the distance is a simple function of the
order statistics In statistics, the ''k''th order statistic of a statistical sample is equal to its ''k''th-smallest value. Together with rank statistics, order statistics are among the most fundamental tools in non-parametric statistics and inference. Important ...
: W_p(P, Q) = \left( \frac\sum_^n \, X_ - Y_\, ^p \right)^\frac.


=Higher dimensions

= If P and Q are empirical distributions, each based on n observations, then W_p(P, Q) = \inf_\pi \left( \frac \sum_^n \, X_i - Y_\, ^p \right)^\frac, where the infimum is over all permutations \pi of n elements. This is a linear assignment problem, and can be solved by the
Hungarian algorithm The Hungarian method is a combinatorial optimization algorithm In mathematics and computer science, an algorithm () is a finite sequence of Rigour#Mathematics, mathematically rigorous instructions, typically used to solve a class of specific ...
in cubic time.


Normal distributions

Let \mu_1 = \mathcal(m_1, C_1) and \mu_2 = \mathcal(m_2, C_2) be two non-degenerate
Gaussian measure In mathematics, Gaussian measure is a Borel measure on finite-dimensional Euclidean space \mathbb^n, closely related to the normal distribution in statistics. There is also a generalization to infinite-dimensional spaces. Gaussian measures are na ...
s (i.e.
normal distribution In probability theory and statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is f(x) = \frac ...
s) on \mathbb^n, with respective
expected value In probability theory, the expected value (also called expectation, expectancy, expectation operator, mathematical expectation, mean, expectation value, or first Moment (mathematics), moment) is a generalization of the weighted average. Informa ...
s m_1 and m_2 \in \mathbb^n and symmetric positive semi-definite
covariance matrices In probability theory and statistics, a covariance matrix (also known as auto-covariance matrix, dispersion matrix, variance matrix, or variance–covariance matrix) is a square matrix giving the covariance between each pair of elements of a ...
C_ and C_2 \in \mathbb^. Then, with respect to the usual Euclidean norm on \mathbb^, the 2-Wasserstein distance between \mu_ and \mu_ is W_ (\mu_1, \mu_2)^2 = \, m_1 - m_2 \, _2^2 + \mathop \left( C_1 + C_2 - 2 \left( C_2^\frac12 C_1 C_2^\frac12 \right)^\frac12 \right) . where C^\frac12 denotes the principal square root of C. Note that the second term (involving the trace) is precisely the (unnormalised)
Bures metric In mathematics, in the area of quantum information geometry, the Bures metric (named after Donald Bures) or Helstrom metric (named after Carl W. Helstrom) defines an infinitesimal distance between density matrix operators defining quantum states. ...
between C_1 and C_2. This result generalises the earlier example of the Wasserstein distance between two point masses (at least in the case p = 2), since a point mass can be regarded as a normal distribution with covariance matrix equal to zero, in which case the
trace Trace may refer to: Arts and entertainment Music * ''Trace'' (Son Volt album), 1995 * ''Trace'' (Died Pretty album), 1993 * Trace (band), a Dutch progressive rock band * ''The Trace'' (album), by Nell Other uses in arts and entertainment * ...
term disappears and only the term involving the Euclidean distance between the means remains.


One-dimensional distributions

Let \mu_1, \mu_2 \in P_p(\mathbb) be probability measures on \mathbb, and denote their cumulative distribution functions by F_1(x) and F_2(x). Then the transport problem has an analytic solution: Optimal transport preserves the order of probability mass elements, so the mass at quantile q of \mu_1 moves to quantile q of \mu_2. Thus, the p-Wasserstein distance between \mu_1 and \mu_2 is W_p(\mu_1, \mu_2) = \left(\int_0^1 \left, F_1^(q) - F_2^(q) \^p \, \mathrm q\right)^\frac, where F_1^ and F_2^ are the quantile functions (inverse CDFs). In the case of p=1, a change of variables leads to the formula W_1(\mu_1, \mu_2) = \int_ \left, F_1(x) - F_2(x) \ \, \mathrm x.


Applications

The Wasserstein metric is a natural way to compare the probability distributions of two variables ''X'' and ''Y'', where one variable is derived from the other by small, non-uniform perturbations (random or deterministic). In computer science, for example, the metric ''W''1 is widely used to compare discrete distributions, ''e.g.'' the
color histogram In image processing and photography, a color histogram is a representation of the distribution of colors in an image. For digital images, a color histogram represents the number of pixels that have colors in each of a fixed list of color ranges t ...
s of two
digital images A digital image is an image composed of picture elements, also known as pixels, each with '' finite'', '' discrete quantities'' of numeric representation for its intensity or gray level that is an output from its two-dimensional functions f ...
; see
earth mover's distance In computer science, the earth mover's distance (EMD) is a measure of dissimilarity between two frequency distributions, densities, or measures, over a metric space ''D''. Informally, if the distributions are interpreted as two different ways of ...
for more details. In their paper '
Wasserstein GAN The Wasserstein Generative Adversarial Network (WGAN) is a variant of generative adversarial network, generative adversarial network (GAN) proposed in 2017 that aims to "improve the stability of learning, get rid of problems like mode collapse, an ...
', Arjovsky et al. use the Wasserstein-1 metric as a way to improve the original framework of
generative adversarial network A generative adversarial network (GAN) is a class of machine learning frameworks and a prominent framework for approaching generative artificial intelligence. The concept was initially developed by Ian Goodfellow and his colleagues in June ...
s (GAN), to alleviate the vanishing gradient and the
mode collapse In machine learning, mode collapse is a failure mode observed in generative models, originally noted in Generative Adversarial Networks (GANs). It occurs when the model produces outputs that are less diverse than expected, effectively "collapsing ...
issues. The special case of normal distributions is used in a Frechet inception distance. The Wasserstein metric has a formal link with
Procrustes analysis In statistics, Procrustes analysis is a form of statistical shape analysis used to analyse the distribution of a set of shapes. The name '' Procrustes'' () refers to a bandit from Greek mythology who made his victims fit his bed either by stretch ...
, with application to chirality measures, and to shape analysis. In computational biology, Wasserstein metric can be used to compare between persistence diagrams of cytometry datasets. The Wasserstein metric also has been used in inverse problems in geophysics. The Wasserstein metric is used in
integrated information theory Integrated information theory (IIT) proposes a mathematical model for the consciousness of a system. It comprises a framework ultimately intended to explain why some physical systems (such as human brains) are conscious, and to be capable of pr ...
to compute the difference between concepts and conceptual structures. The Wasserstein metric and related formulations have also been used to provide a unified theory for shape observable analysis in high energy and collider physics datasets.


Properties


Metric structure

It can be shown that ''W''''p'' satisfies all the
axiom An axiom, postulate, or assumption is a statement that is taken to be true, to serve as a premise or starting point for further reasoning and arguments. The word comes from the Ancient Greek word (), meaning 'that which is thought worthy or ...
s of a
metric Metric or metrical may refer to: Measuring * Metric system, an internationally adopted decimal system of measurement * An adjective indicating relation to measurement in general, or a noun describing a specific type of measurement Mathematics ...
on the Wasserstein space ''P''''p''(''M'') consisting of all Borel probability measures on ''M'' having finite ''p''th moment. Furthermore, convergence with respect to ''W''''p'' is equivalent to the usual
weak convergence of measures In mathematics, more specifically measure theory, there are various notions of the convergence of measures. For an intuitive general sense of what is meant by ''convergence of measures'', consider a sequence of measures on a space, sharing a com ...
plus convergence of the first ''p''th moments.


Dual representation of ''W''1

The following dual representation of ''W''1 is a special case of the duality theorem of
Kantorovich Leonid Vitalyevich Kantorovich (, ; 19 January 19127 April 1986) was a Soviet mathematician and economist, known for his theory and development of techniques for the optimal allocation of resources. He is regarded as the founder of linear programm ...
and Rubinstein (1958): when ''μ'' and ''ν'' have bounded
support Support may refer to: Arts, entertainment, and media * Supporting character * Support (art), a solid surface upon which a painting is executed Business and finance * Support (technical analysis) * Child support * Customer support * Income Su ...
, W_1 (\mu, \nu) = \sup \left\, where Lip(''f'') denotes the minimal
Lipschitz constant In mathematical analysis, Lipschitz continuity, named after German mathematician Rudolf Lipschitz, is a strong form of uniform continuity for functions. Intuitively, a Lipschitz continuous function is limited in how fast it can change: there exis ...
for ''f''. This form shows that ''W''1 is an integral probability metric. Compare this with the definition of the
Radon metric In mathematics (specifically in measure theory), a Radon measure, named after Johann Radon, is a measure on the -algebra of Borel sets of a Hausdorff topological space that is finite on all compact sets, outer regular on all Borel sets, and i ...
: \rho (\mu, \nu) := \sup \left\. If the metric ''d'' of the metric space (''M'',''d'') is bounded by some constant ''C'', then 2 W_1 (\mu, \nu) \leq C \rho (\mu, \nu), and so convergence in the Radon metric (identical to total variation convergence when ''M'' is a
Polish space In the mathematical discipline of general topology, a Polish space is a separable space, separable Completely metrizable space, completely metrizable topological space; that is, a space homeomorphic to a Complete space, complete metric space that h ...
) implies convergence in the Wasserstein metric, but not vice versa.


Proof

The following is an intuitive proof which skips over technical points. A fully rigorous proof is found in. Discrete case: When M is discrete, solving for the 1-Wasserstein distance is a problem in linear programming: \begin \min_\gamma \displaystyle \sum_ c(x, y) \gamma(x, y) \\ pt \displaystyle \sum_y \gamma(x, y) = \mu(x) \\ pt \displaystyle \sum_x \gamma(x, y) = \nu(y) \\ pt \gamma \geq 0 \end where c: M \times M \to
dual problem In mathematical optimization theory, duality or the duality principle is the principle that optimization problems may be viewed from either of two perspectives, the primal problem or the dual problem. If the primal is a minimization problem then th ...
: \begin \max_ \displaystyle \sum_x \mu(x)f(x) + \sum_y \nu(y)g(y)\\ pt f(x) + g(y) \leq c(x, y) \end and by the Dual linear program#Strong duality">duality theorem of linear programming, since the primal problem is feasible and bounded, so is the dual problem, and the minimum in the first problem equals the maximum in the second problem. That is, the problem pair exhibits ''strong duality''. For the general case, the dual problem is found by converting sums to integrals: \begin \sup_ \mathbb E_ (x)+ \mathbb E_ (y)\ pt f(x) + g(y) \leq c(x, y) \end and the ''strong duality'' still holds. This is the Kantorovich duality theorem.
Cédric Villani Cédric Patrice Thierry Villani (; born 5 October 1973) is a French politician and mathematician working primarily on partial differential equations, Riemannian geometry and mathematical physics. He was awarded the Fields Medal in 2010, and he ...
recounts the following interpretation from
Luis Caffarelli Luis Ángel Caffarelli (; born December 8, 1948) is an Argentine-American mathematician. He studies partial differential equations and their applications. Caffarelli is a professor of mathematics at the University of Texas at Austin, and the win ...
:
Suppose you want to ship some coal from mines, distributed as \mu, to factories, distributed as \nu. The cost function of transport is c. Now a shipper comes and offers to do the transport for you. You would pay him f(x) per coal for loading the coal at x, and pay him g(y) per coal for unloading the coal at y. For you to accept the deal, the price schedule must satisfy f(x) + g(y) \leq c(x, y). The Kantorovich duality states that the shipper can make a price schedule that makes you pay almost as much as you would ship yourself.
This result can be pressed further to yield: The two infimal convolution steps are visually clear when the probability space is \R. For notational convenience, let \square denote the infimal convolution operation. For the first step, where we used f = \text \mathbin (-g), plot out the curve of -g, then at each point, draw a cone of slope 1, and take the lower envelope of the cones as f, as shown in the diagram, then f cannot increase with slope larger than 1. Thus all its secants have slope \bigg, \frac\bigg, \leq 1. For the second step, picture the infimal convolution \text \mathbin (-f), then if all secants of f have slope at most 1, then the lower envelope of \text \mathbin (-f) are just the cone-apices themselves, thus \text \mathbin (-f)=-f. 1D Example. When both \mu, \nu are distributions on \R, then integration by parts give \mathbb_ (x)- \mathbb E_ (y)= \int f'(x) (F_\nu(x) - F_\mu(x)) \, \mathrmx, thus f(x) = K \cdot \operatorname(F_\nu(x) - F_\mu(x)).


Fluid mechanics interpretation of ''W''2

Benamou & Brenier found a dual representation of W_2 by
fluid mechanics Fluid mechanics is the branch of physics concerned with the mechanics of fluids (liquids, gases, and plasma (physics), plasmas) and the forces on them. Originally applied to water (hydromechanics), it found applications in a wide range of discipl ...
, which allows efficient solution by
convex optimization Convex optimization is a subfield of mathematical optimization that studies the problem of minimizing convex functions over convex sets (or, equivalently, maximizing concave functions over convex sets). Many classes of convex optimization problems ...
. Given two probability densities p, q on \R^n, W_2^2(p, q)=\min_ \int_0^1 \int_ \, \bold(\bold, t)\, ^2 \rho(\bold, t) \, d \bold \, dt where \bold ranges over velocity fields driving the
continuity equation A continuity equation or transport equation is an equation that describes the transport of some quantity. It is particularly simple and powerful when applied to a conserved quantity, but it can be generalized to apply to any extensive quantity ...
with boundary conditions on the fluid density field: \dot+\nabla\cdot(\rho\bold)=0 \quad \rho(\cdot, 0)=p,\; \rho(\cdot, 1)=q That is, the mass should be conserved, and the velocity field should transport the probability distribution p to q during the time interval
, 1 The comma is a punctuation mark that appears in several variants in different languages. Some typefaces render it as a small line, slightly curved or straight, but inclined from the vertical; others give it the appearance of a miniature fille ...
/math>.


Equivalence of ''W''2 and a negative-order Sobolev norm

Under suitable assumptions, the Wasserstein distance W_2 of order two is Lipschitz equivalent to a negative-order homogeneous Sobolev norm. More precisely, if we take M to be a
connected Connected may refer to: Film and television * ''Connected'' (2008 film), a Hong Kong remake of the American movie ''Cellular'' * '' Connected: An Autoblogography About Love, Death & Technology'', a 2011 documentary film * ''Connected'' (2015 TV ...
Riemannian manifold In differential geometry, a Riemannian manifold is a geometric space on which many geometric notions such as distance, angles, length, volume, and curvature are defined. Euclidean space, the N-sphere, n-sphere, hyperbolic space, and smooth surf ...
equipped with a positive measure \pi, then we may define for f \colon M \to \mathbb the seminorm \, f \, _^ = \int_ \, \nabla f(x)\, ^ \, \pi(\mathrm x) and for a
signed measure In mathematics, a signed measure is a generalization of the concept of (positive) measure by allowing the set function to take negative values, i.e., to acquire sign. Definition There are two slightly different concepts of a signed measure, de ...
\mu on M the dual norm \, \mu \, _ = \sup \bigg\ . Then any two probability measures \mu and \nu on M satisfy the upper bound W_ (\mu, \nu) \leq 2\, \, \mu - \nu \, _ . In the other direction, if \mu and \nu each have densities with respect to the standard volume measure on M that are both bounded above by some 0 < C < \infty, and M has non-negative
Ricci curvature In differential geometry, the Ricci curvature tensor, named after Gregorio Ricci-Curbastro, is a geometric object which is determined by a choice of Riemannian or pseudo-Riemannian metric on a manifold. It can be considered, broadly, as a measure ...
, then \, \mu - \nu \, _ \leq \sqrt\, W_ (\mu, \nu) .


Separability and completeness

For any ''p'' ≥ 1, the metric space (''P''''p''(''M''), ''W''''p'') is separable, and is
complete Complete may refer to: Logic * Completeness (logic) * Completeness of a theory, the property of a theory that every formula in the theory's language or its negation is provable Mathematics * The completeness of the real numbers, which implies t ...
if (''M'', ''d'') is separable and complete.


Wasserstein distance for ''p'' = ∞

It is also possible to consider the Wasserstein metric for p = \infty. In this case, the defining formula becomes: W_(\mu,\nu) = \lim_ W_p(\mu,\nu) = \inf_ \gamma\operatorname d(x,y), where \gamma\operatorname d(x,y) denotes the
essential supremum In mathematics, the concepts of essential infimum and essential supremum are related to the notions of infimum and supremum, but adapted to measure theory and functional analysis, where one often deals with statements that are not valid for ''all' ...
of d(x,y) with respect to measure \gamma. The metric space (''P''(''M''), ''W'') is complete if (''M'', ''d'') is separable and complete. Here, ''P'' is the space of all probability measures with bounded support.


See also

*
Hutchinson metric In mathematics, the Hutchinson metric otherwise known as Kantorovich metric is a function which measures "the discrepancy between two images for use in fractal image processing" and "can also be applied to describe the similarity between DNA seq ...
*
Lévy metric In mathematics, the Lévy metric is a metric on the space of cumulative distribution functions of one-dimensional random variables. It is a special case of the Lévy–Prokhorov metric, and is named after the French mathematician Paul Lévy. Def ...
*
Lévy–Prokhorov metric In mathematics, the Lévy–Prokhorov metric (sometimes known just as the Prokhorov metric) is a metric (i.e., a definition of distance) on the collection of probability measures on a given metric space. It is named after the French mathematician P ...
*
Fréchet distance In mathematics, the Fréchet distance is a measure of similarity between curves that takes into account the location and ordering of the points along the curves. It is named after Maurice Fréchet. Intuitive definition Imagine a person traversing ...
*
Total variation distance of probability measures In probability theory, the total variation distance is a statistical distance between probability distributions, and is sometimes called the statistical distance, statistical difference or variational distance. Definition Consider a measurab ...
* Transportation theory *
Earth mover's distance In computer science, the earth mover's distance (EMD) is a measure of dissimilarity between two frequency distributions, densities, or measures, over a metric space ''D''. Informally, if the distributions are interpreted as two different ways of ...
*
Wasserstein GAN The Wasserstein Generative Adversarial Network (WGAN) is a variant of generative adversarial network, generative adversarial network (GAN) proposed in 2017 that aims to "improve the stability of learning, get rid of problems like mode collapse, an ...
*
Kolmogorov–Smirnov test In statistics, the Kolmogorov–Smirnov test (also K–S test or KS test) is a nonparametric statistics, nonparametric test of the equality of continuous (or discontinuous, see #Discrete and mixed null distribution, Section 2.2), one-dimensional ...


References


Further reading

* * * *


External links

* {{cite web , url=https://stats.stackexchange.com/q/295617 , title=What is the advantages of Wasserstein metric compared to Kullback–Leibler divergence? , date=August 1, 2017 , work=
Stack Exchange Stack Exchange is a network of question-and-answer (Q&A) websites on topics in diverse fields, each site covering a specific topic, where questions, answers, and users are subject to a reputation award process. The reputation system allows t ...
Measure theory Metric geometry Theory of probability distributions Statistical distance