In
probability theory
Probability theory or probability calculus is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expre ...
and
statistics
Statistics (from German language, German: ', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a s ...
, the exponential distribution or negative exponential distribution is the
probability distribution
In probability theory and statistics, a probability distribution is a Function (mathematics), function that gives the probabilities of occurrence of possible events for an Experiment (probability theory), experiment. It is a mathematical descri ...
of the distance between events in a
Poisson point process
In probability theory, statistics and related fields, a Poisson point process (also known as: Poisson random measure, Poisson random point field and Poisson point field) is a type of mathematical object that consists of points randomly located ...
, i.e., a process in which events occur continuously and independently at a constant average rate; the distance parameter could be any meaningful mono-dimensional measure of the process, such as time between production errors, or length along a roll of fabric in the weaving manufacturing process. It is a particular case of the
gamma distribution
In probability theory and statistics, the gamma distribution is a versatile two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-squared distribution are special cases of the g ...
. It is the continuous analogue of the
geometric distribution
In probability theory and statistics, the geometric distribution is either one of two discrete probability distributions:
* The probability distribution of the number X of Bernoulli trials needed to get one success, supported on \mathbb = \;
* T ...
, and it has the key property of being
memoryless
In probability and statistics, memorylessness is a property of probability distributions. It describes situations where previous failures or elapsed time does not affect future trials or further wait time. Only the geometric and exponential d ...
. In addition to being used for the analysis of Poisson point processes it is found in various other contexts.
The exponential distribution is not the same as the class of
exponential families
In probability and statistics, an exponential family is a parametric set of probability distributions of a certain form, specified below. This special form is chosen for mathematical convenience, including the enabling of the user to calculate ...
of distributions. This is a large class of probability distributions that includes the exponential distribution as one of its members, but also includes many other distributions, like the
normal,
binomial
Binomial may refer to:
In mathematics
*Binomial (polynomial), a polynomial with two terms
*Binomial coefficient, numbers appearing in the expansions of powers of binomials
*Binomial QMF, a perfect-reconstruction orthogonal wavelet decomposition
* ...
,
gamma
Gamma (; uppercase , lowercase ; ) is the third letter of the Greek alphabet. In the system of Greek numerals it has a value of 3. In Ancient Greek, the letter gamma represented a voiced velar stop . In Modern Greek, this letter normally repr ...
, and
Poisson distributions.
[
]
Definitions
Probability density function
The probability density function
In probability theory, a probability density function (PDF), density function, or density of an absolutely continuous random variable, is a Function (mathematics), function whose value at any given sample (or point) in the sample space (the s ...
(pdf) of an exponential distribution is
:
Here ''λ'' > 0 is the parameter of the distribution, often called the ''rate parameter''. The distribution is supported on the interval . If a random variable
A random variable (also called random quantity, aleatory variable, or stochastic variable) is a Mathematics, mathematical formalization of a quantity or object which depends on randomness, random events. The term 'random variable' in its mathema ...
''X'' has this distribution, we write .
The exponential distribution exhibits infinite divisibility
Infinite divisibility arises in different ways in philosophy, physics, economics, order theory (a branch of mathematics), and probability theory (also a branch of mathematics). One may speak of infinite divisibility, or the lack thereof, of matter ...
.
Cumulative distribution function
The cumulative distribution function
In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x.
Ever ...
is given by
:
Alternative parametrization
The exponential distribution is sometimes parametrized in terms of the scale parameter
In probability theory and statistics, a scale parameter is a special kind of numerical parameter of a parametric family of probability distributions. The larger the scale parameter, the more spread out the distribution.
Definition
If a family ...
, which is also the mean:
Properties
Mean, variance, moments, and median
The mean or expected value
In probability theory, the expected value (also called expectation, expectancy, expectation operator, mathematical expectation, mean, expectation value, or first Moment (mathematics), moment) is a generalization of the weighted average. Informa ...
of an exponentially distributed random variable ''X'' with rate parameter ''λ'' is given by
In light of the examples given below
Below may refer to:
*Earth
*Ground (disambiguation)
*Soil
*Floor
* Bottom (disambiguation)
*Less than
*Temperatures below freezing
*Hell or underworld
People with the surname
* Ernst von Below (1863–1955), German World War I general
* Fred Belo ...
, this makes sense; a person who receives an average of two telephone calls per hour can expect that the time between consecutive calls will be 0.5 hour, or 30 minutes.
The variance
In probability theory and statistics, variance is the expected value of the squared deviation from the mean of a random variable. The standard deviation (SD) is obtained as the square root of the variance. Variance is a measure of dispersion ...
of ''X'' is given by
so the standard deviation
In statistics, the standard deviation is a measure of the amount of variation of the values of a variable about its Expected value, mean. A low standard Deviation (statistics), deviation indicates that the values tend to be close to the mean ( ...
is equal to the mean.
The moments of ''X'', for are given by
The central moment
In probability theory and statistics, a central moment is a moment of a probability distribution of a random variable about the random variable's mean; that is, it is the expected value of a specified integer power of the deviation of the random ...
s of ''X'', for are given by
where !''n'' is the subfactorial
In combinatorial mathematics, a derangement is a permutation of the elements of a set in which no element appears in its original position. In other words, a derangement is a permutation that has no fixed points.
The number of derangements of ...
of ''n''
The median
The median of a set of numbers is the value separating the higher half from the lower half of a Sample (statistics), data sample, a statistical population, population, or a probability distribution. For a data set, it may be thought of as the “ ...
of ''X'' is given by
where refers to the natural logarithm
The natural logarithm of a number is its logarithm to the base of a logarithm, base of the e (mathematical constant), mathematical constant , which is an Irrational number, irrational and Transcendental number, transcendental number approxima ...
. Thus the absolute difference
The absolute difference of two real numbers x and y is given by , x-y, , the absolute value of their difference. It describes the distance on the real line between the points corresponding to x and y, and is a special case of the Lp distance fo ...
between the mean and median is
in accordance with the median-mean inequality
In probability theory, Chebyshev's inequality (also called the Bienaymé–Chebyshev inequality) provides an upper bound on the probability of deviation of a random variable (with finite variance) from its mean. More specifically, the probability ...
.
Memorylessness property of exponential random variable
An exponentially distributed random variable ''T'' obeys the relation
This can be seen by considering the complementary cumulative distribution function
In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x.
Ever ...
:
When ''T'' is interpreted as the waiting time for an event to occur relative to some initial time, this relation implies that, if ''T'' is conditioned on a failure to observe the event over some initial period of time ''s'', the distribution of the remaining waiting time is the same as the original unconditional distribution. For example, if an event has not occurred after 30 seconds, the conditional probability
In probability theory, conditional probability is a measure of the probability of an Event (probability theory), event occurring, given that another event (by assumption, presumption, assertion or evidence) is already known to have occurred. This ...
that occurrence will take at least 10 more seconds is equal to the unconditional probability of observing the event more than 10 seconds after the initial time.
The exponential distribution and the geometric distribution
In probability theory and statistics, the geometric distribution is either one of two discrete probability distributions:
* The probability distribution of the number X of Bernoulli trials needed to get one success, supported on \mathbb = \;
* T ...
are the only memoryless probability distributions.
The exponential distribution is consequently also necessarily the only continuous probability distribution that has a constant failure rate
Failure is the social concept of not meeting a desirable or intended objective, and is usually viewed as the opposite of success. The criteria for failure depends on context, and may be relative to a particular observer or belief system. On ...
.
Quantiles
The quantile function
In probability and statistics, the quantile function is a function Q: ,1\mapsto \mathbb which maps some probability x \in ,1/math> of a random variable v to the value of the variable y such that P(v\leq y) = x according to its probability distr ...
(inverse cumulative distribution function) for Exp(''λ'') is
The quartile
In statistics, quartiles are a type of quantiles which divide the number of data points into four parts, or ''quarters'', of more-or-less equal size. The data must be ordered from smallest to largest to compute quartiles; as such, quartiles are ...
s are therefore:
* first quartile: ln(4/3)/''λ''
* median
The median of a set of numbers is the value separating the higher half from the lower half of a Sample (statistics), data sample, a statistical population, population, or a probability distribution. For a data set, it may be thought of as the “ ...
: ln(2)/''λ''
* third quartile: ln(4)/''λ''
And as a consequence the interquartile range
In descriptive statistics, the interquartile range (IQR) is a measure of statistical dispersion, which is the spread of the data. The IQR may also be called the midspread, middle 50%, fourth spread, or H‑spread. It is defined as the differen ...
is ln(3)/''λ''.
Conditional Value at Risk (Expected Shortfall)
The conditional value at risk (CVaR) also known as the expected shortfall
Expected shortfall (ES) is a risk measure—a concept used in the field of financial risk measurement to evaluate the market risk or credit risk of a portfolio. The "expected shortfall at q% level" is the expected return on the portfolio in the wor ...
or superquantile for Exp(''λ'') is derived as follows:
Buffered Probability of Exceedance (bPOE)
The buffered probability of exceedance is one minus the probability level at which the CVaR equals the threshold . It is derived as follows:[
]
Kullback–Leibler divergence
The directed Kullback–Leibler divergence
In mathematical statistics, the Kullback–Leibler (KL) divergence (also called relative entropy and I-divergence), denoted D_\text(P \parallel Q), is a type of statistical distance: a measure of how much a model probability distribution is diff ...
in nats of ("approximating" distribution) from ('true' distribution) is given by
Maximum entropy distribution
Among all continuous probability distributions with support
Support may refer to:
Arts, entertainment, and media
* Supporting character
* Support (art), a solid surface upon which a painting is executed
Business and finance
* Support (technical analysis)
* Child support
* Customer support
* Income Su ...
and mean ''μ'', the exponential distribution with ''λ'' = 1/''μ'' has the largest differential entropy
Differential entropy (also referred to as continuous entropy) is a concept in information theory that began as an attempt by Claude Shannon to extend the idea of (Shannon) entropy (a measure of average surprisal) of a random variable, to continu ...
. In other words, it is the maximum entropy probability distribution
In statistics and information theory, a maximum entropy probability distribution has entropy that is at least as great as that of all other members of a specified class of probability distributions. According to the principle of maximum entropy, ...
for a random variate
In probability and statistics, a random variate or simply variate is a particular outcome or ''realization'' of a random variable; the random variates which are other outcomes of the same random variable might have different values ( random numbe ...
''X'' which is greater than or equal to zero and for which E 'X''is fixed.
Distribution of the minimum of exponential random variables
Let ''X''1, ..., ''X''''n'' be independent
Independent or Independents may refer to:
Arts, entertainment, and media Artist groups
* Independents (artist group), a group of modernist painters based in Pennsylvania, United States
* Independentes (English: Independents), a Portuguese artist ...
exponentially distributed random variables with rate parameters ''λ''1, ..., ''λn''. Then
is also exponentially distributed, with parameter
This can be seen by considering the complementary cumulative distribution function
In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x.
Ever ...
:
The index of the variable which achieves the minimum is distributed according to the categorical distribution
A proof can be seen by letting . Then,
Note that
is not exponentially distributed, if ''X''1, ..., ''X''''n'' do not all have parameter 0.
Joint moments of i.i.d. exponential order statistics
Let be independent and identically distributed
Independent or Independents may refer to:
Arts, entertainment, and media Artist groups
* Independents (artist group), a group of modernist painters based in Pennsylvania, United States
* Independentes (English: Independents), a Portuguese artist ...
exponential random variables with rate parameter ''λ''.
Let denote the corresponding order statistic
In statistics, the ''k''th order statistic of a statistical sample is equal to its ''k''th-smallest value. Together with Ranking (statistics), rank statistics, order statistics are among the most fundamental tools in non-parametric statistics and ...
s.
For , the joint moment of the order statistics and is given by
This can be seen by invoking the law of total expectation
The proposition in probability theory known as the law of total expectation, the law of iterated expectations (LIE), Adam's law, the tower rule, and the smoothing property of conditional expectation, among other names, states that if X is a random ...
and the memoryless property:
The first equation follows from the law of total expectation
The proposition in probability theory known as the law of total expectation, the law of iterated expectations (LIE), Adam's law, the tower rule, and the smoothing property of conditional expectation, among other names, states that if X is a random ...
.
The second equation exploits the fact that once we condition on , it must follow that . The third equation relies on the memoryless property to replace
Sum of two independent exponential random variables
The probability distribution function (PDF) of a sum of two independent random variables is the convolution of their individual PDFs. If X_1 and X_2 are independent exponential random variables with respective rate parameters \lambda_1 and \lambda_2, then the probability density of Z=X_1+X_2 is given by
\begin
f_Z(z) &= \int_^\infty f_(x_1) f_(z - x_1)\,dx_1\\
&= \int_0^z \lambda_1 e^ \lambda_2 e^ \, dx_1 \\
&= \lambda_1 \lambda_2 e^ \int_0^z e^\,dx_1 \\
&= \begin
\dfrac \left(e^ - e^\right) & \text \lambda_1 \neq \lambda_2 \\ pt \lambda^2 z e^ & \text \lambda_1 = \lambda_2 = \lambda.
\end
\end
The entropy of this distribution is available in closed form: assuming \lambda_1 > \lambda_2 (without loss of generality), then
\begin
H(Z) &= 1 + \gamma + \ln \left( \frac \right) + \psi \left( \frac \right) ,
\end
where \gamma is the Euler-Mascheroni constant
Euler's constant (sometimes called the Euler–Mascheroni constant) is a mathematical constant, usually denoted by the lowercase Greek letter gamma (), defined as the limiting difference between the harmonic series and the natural logarith ...
, and \psi(\cdot) is the digamma function
In mathematics, the digamma function is defined as the logarithmic derivative of the gamma function:
:\psi(z) = \frac\ln\Gamma(z) = \frac.
It is the first of the polygamma functions. This function is Monotonic function, strictly increasing a ...
.
In the case of equal rate parameters, the result is an Erlang distribution
The Erlang distribution is a two-parameter family of continuous probability distributions with Support (mathematics), support x \in with shape 2 and parameter \lambda, which in turn is a special case of gamma distribution
In probability theory and statistics, the gamma distribution is a versatile two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-squared distribution are special cases of the g ...
.
The sum of n independent Exp(''λ)'' exponential random variables is Gamma(n, ''λ)'' distributed.
Related distributions
* If ''X'' ~ Laplace(μ, β−1), then ">''X'' − μ, ~ Exp(β).[
* If ''X'' ~ ''U''(0, 1) then −log(''X'') ~ Exp(1).
* If ''X'' ~ Pareto(1, λ), then log(''X'') ~ Exp(λ).]
* If ''X'' ~ SkewLogistic(θ), then \log\left(1 + e^\right) \sim \operatorname(\theta).
* If ''Xi'' ~ ''U''(0, 1) then \lim_n \min \left(X_1, \ldots, X_n\right) \sim \operatorname(1)
* The exponential distribution is a limit of a scaled : \lim_ n \operatorname(1, n) = \operatorname(1).
* The exponential distribution is a special case of type 3 Pearson distribution">beta distribution: \lim_ n \operatorname(1, n) = \operatorname(1).
* The exponential distribution is a special case of type 3 Pearson distribution.
* The exponential distribution is the special case of a Gamma distribution
In probability theory and statistics, the gamma distribution is a versatile two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-squared distribution are special cases of the g ...
with shape parameter 1.[
* If ''X'' ~ Exp(λ) and ''X'' ~ Exp(λ) then:
** kX \sim \operatorname\left(\frac\right), closure under scaling by a positive factor.
** 1 + ''X'' ~ BenktanderWeibull(λ, 1), which reduces to a truncated exponential distribution.
** ''keX'' ~ Pareto(''k'', λ).][
** ''e−λX'' ~ ''U''(0, 1).
** ''e−X'' ~ ]Beta
Beta (, ; uppercase , lowercase , or cursive ; or ) is the second letter of the Greek alphabet. In the system of Greek numerals, it has a value of 2. In Ancient Greek, beta represented the voiced bilabial plosive . In Modern Greek, it represe ...
(λ, 1).[
** ''e'' ~ PowerLaw(''k'', λ)
** \sqrt \sim \operatorname \left(\frac\right), the ]Rayleigh distribution
In probability theory and statistics, the Rayleigh distribution is a continuous probability distribution for nonnegative-valued random variables. Up to rescaling, it coincides with the chi distribution with two degrees of freedom.
The distributi ...
[
** X \sim \operatorname\left(\frac, 1\right), the ]Weibull distribution
In probability theory and statistics, the Weibull distribution is a continuous probability distribution. It models a broad range of random variables, largely in the nature of a time to failure or time between events. Examples are maximum on ...
[
** X^2 \sim \operatorname\left(\frac, \frac\right)][
** .
** \lfloor X\rfloor \sim \operatorname\left(1-e^\right), a ]geometric distribution
In probability theory and statistics, the geometric distribution is either one of two discrete probability distributions:
* The probability distribution of the number X of Bernoulli trials needed to get one success, supported on \mathbb = \;
* T ...
on 0,1,2,3,...
** \lceil X\rceil \sim \operatorname\left(1-e^\right), a geometric distribution
In probability theory and statistics, the geometric distribution is either one of two discrete probability distributions:
* The probability distribution of the number X of Bernoulli trials needed to get one success, supported on \mathbb = \;
* T ...
on 1,2,3,4,...
** If also ''Y'' ~ Erlang(''n'', λ) orY \sim \Gamma\left(n, \frac\right) then \frac + 1 \sim \operatorname(1, n)
** If also λ ~ Gamma
Gamma (; uppercase , lowercase ; ) is the third letter of the Greek alphabet. In the system of Greek numerals it has a value of 3. In Ancient Greek, the letter gamma represented a voiced velar stop . In Modern Greek, this letter normally repr ...
(''k'', θ) (shape, scale parametrisation) then the marginal distribution of ''X'' is Lomax(''k'', 1/θ), the gamma mixture
In chemistry, a mixture is a material made up of two or more different chemical substances which can be separated by physical method. It is an impure substance made up of 2 or more elements or compounds mechanically mixed together in any proporti ...
** λ''X'' − λ''Y'' ~ Laplace(0, 1).
** min ~ Exp(λ1 + ... + λ''n'').
** If also λ = λ then:
*** X_1 + \cdots + X_k = \sum_i X_i \sim Erlang(''k'', λ) = Gamma
Gamma (; uppercase , lowercase ; ) is the third letter of the Greek alphabet. In the system of Greek numerals it has a value of 3. In Ancient Greek, the letter gamma represented a voiced velar stop . In Modern Greek, this letter normally repr ...
(''k'', λ) with integer shape parameter ''k'' and rate parameter λ.
*** If T = (X_1 + \cdots + X_n ) = \sum_^n X_i, then 2 \lambda T \sim \chi^2_.
*** ''X'' − ''X'' ~ Laplace(0, λ−1).
** If also ''X'' are independent, then:
*** \frac ~ U(0, 1)
*** Z = \frac has probability density function f_Z(z) = \frac. This can be used to obtain a confidence interval for \frac.
** If also λ = 1:
*** \mu - \beta\log\left(\frac\right) \sim \operatorname(\mu, \beta), the logistic distribution
In probability theory and statistics, the logistic distribution is a continuous probability distribution. Its cumulative distribution function is the logistic function, which appears in logistic regression and feedforward neural networks. It rese ...
*** \mu - \beta\log\left(\frac\right) \sim \operatorname(\mu, \beta)
*** ''μ'' − σ log(''X'') ~ GEV(μ, σ, 0).
*** Further if Y \sim \Gamma\left(\alpha, \frac\right) then \sqrt \sim \operatorname(\alpha, \beta) (K-distribution
In probability and statistics, the generalized K-distribution is a three-parameter family of continuous probability distributions.
The distribution arises by compounding two gamma distributions. In each case, a re-parametrization of the usual f ...
)
** If also λ = 1/2 then ; i.e., ''X'' has a chi-squared distribution
In probability theory and statistics, the \chi^2-distribution with k Degrees of freedom (statistics), degrees of freedom is the distribution of a sum of the squares of k Independence (probability theory), independent standard normal random vari ...
with 2 degrees of freedom
In many scientific fields, the degrees of freedom of a system is the number of parameters of the system that may vary independently. For example, a point in the plane has two degrees of freedom for translation: its two coordinates; a non-infinite ...
. Hence: \operatorname(\lambda) = \frac \operatorname\left(\frac \right) \sim \frac \chi_2^2\Rightarrow \sum_^n \operatorname(\lambda) \sim \frac\chi_^2
* If X \sim \operatorname\left(\frac\right) and Y \mid X ~ Poisson(''X'') then Y \sim \operatorname\left(\frac\right) (geometric distribution
In probability theory and statistics, the geometric distribution is either one of two discrete probability distributions:
* The probability distribution of the number X of Bernoulli trials needed to get one success, supported on \mathbb = \;
* T ...
)
* The Hoyt distribution can be obtained from exponential distribution and arcsine distribution
* The exponential distribution is a limit of the ''κ''-exponential distribution in the \kappa = 0 case.
* Exponential distribution is a limit of the κ-Generalized Gamma distribution in the \alpha = 1 and \nu = 1 cases:
*: \lim_ p_\kappa(x) = (1+\kappa\nu)(2\kappa)^\nu \frac \frac x^\exp_\kappa(-\lambda x^\alpha) = \lambda e^
Other related distributions:
* Hyper-exponential distribution – the distribution whose density is a weighted sum of exponential densities.
* Hypoexponential distribution
In probability theory the hypoexponential distribution or the generalized Erlang distribution is a continuous distribution, that has found use in the same fields as the Erlang distribution, such as queueing theory, teletraffic engineering and mor ...
– the distribution of a general sum of exponential random variables.[
* exGaussian distribution – the sum of an exponential distribution and a ]normal distribution
In probability theory and statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is
f(x) = \frac ...
.
Statistical inference
Below, suppose random variable ''X'' is exponentially distributed with rate parameter λ, and x_1, \dotsc, x_n are ''n'' independent samples from ''X'', with sample mean \bar.
Parameter estimation
The maximum likelihood
In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed stati ...
estimator for λ is constructed as follows.
The likelihood function
A likelihood function (often simply called the likelihood) measures how well a statistical model explains observed data by calculating the probability of seeing that data under different parameter values of the model. It is constructed from the ...
for λ, given an independent and identically distributed
Independent or Independents may refer to:
Arts, entertainment, and media Artist groups
* Independents (artist group), a group of modernist painters based in Pennsylvania, United States
* Independentes (English: Independents), a Portuguese artist ...
sample ''x'' = (''x''1, ..., ''x''''n'') drawn from the variable, is:
L(\lambda) = \prod_^n\lambda\exp(-\lambda x_i) = \lambda^n\exp\left(-\lambda \sum_^n x_i\right) = \lambda^n\exp\left(-\lambda n\overline\right),
where:
\overline = \frac\sum_^n x_i
is the sample mean.
The derivative of the likelihood function's logarithm is:
\frac \ln L(\lambda) =
\frac \left( n \ln\lambda - \lambda n\overline \right) =
\frac - n\overline\ \begin
> 0, & 0 < \lambda < \frac, \\ pt = 0, & \lambda = \frac, \\ pt < 0, & \lambda > \frac.
\end
Consequently, the maximum likelihood
In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed stati ...
estimate for the rate parameter is:
\widehat_\text = \frac = \frac
This is an unbiased estimator
In statistics, the bias of an estimator (or bias function) is the difference between this estimator's expected value and the true value of the parameter being estimated. An estimator or decision rule with zero bias is called ''unbiased''. In stat ...
of \lambda, although \overline an unbiased MLE estimator of 1/\lambda and the distribution mean.
The bias of \widehat_\text is equal to
B \equiv \operatorname\left left(\widehat_\text - \lambda\right)\right= \frac
which yields the bias-corrected maximum likelihood estimator
\widehat^*_\text = \widehat_\text - B.
An approximate minimizer of mean squared error
In statistics, the mean squared error (MSE) or mean squared deviation (MSD) of an estimator (of a procedure for estimating an unobserved quantity) measures the average of the squares of the errors—that is, the average squared difference betwee ...
(see also: bias–variance tradeoff
In statistics and machine learning, the bias–variance tradeoff describes the relationship between a model's complexity, the accuracy of its predictions, and how well it can make predictions on previously unseen data that were not used to train ...
) can be found, assuming a sample size greater than two, with a correction factor to the MLE:
\widehat = \left(\frac\right) \left(\frac\right) = \frac
This is derived from the mean and variance of the inverse-gamma distribution
In probability theory and statistics, the inverse gamma distribution is a two-parameter family of continuous probability distributions on the positive real line, which is the distribution of the reciprocal of a variable distributed according to ...
, \mbox(n, \lambda).
Fisher information
The Fisher information
In mathematical statistics, the Fisher information is a way of measuring the amount of information that an observable random variable ''X'' carries about an unknown parameter ''θ'' of a distribution that models ''X''. Formally, it is the variance ...
, denoted \mathcal(\lambda), for an estimator of the rate parameter \lambda is given as:
\mathcal(\lambda) = \operatorname \left \lambda\right= \int \left(\frac \log f(x;\lambda)\right)^2 f(x; \lambda)\,dx
Plugging in the distribution and solving gives:
\mathcal(\lambda) = \int_^ \left(\frac \log \lambda e^\right)^2 \lambda e^\,dx = \int_^ \left(\frac - x\right)^2 \lambda e^\,dx = \lambda^.
This determines the amount of information each independent sample of an exponential distribution carries about the unknown rate parameter \lambda.
Confidence intervals
An exact 100(1 − α)% confidence interval for the rate parameter of an exponential distribution is given by:
\frac< \frac < \frac\,,
which is also equal to
\frac < \frac < \frac\,,
where is the percentile
In statistics, a ''k''-th percentile, also known as percentile score or centile, is a score (e.g., a data point) a given percentage ''k'' of all scores in its frequency distribution exists ("exclusive" definition) or a score a given percentage ...
of the chi squared distribution
In probability theory and statistics, the \chi^2-distribution with k degrees of freedom is the distribution of a sum of the squares of k independent standard normal random variables.
The chi-squared distribution \chi^2_k is a special case of ...
with ''v'' degrees of freedom
In many scientific fields, the degrees of freedom of a system is the number of parameters of the system that may vary independently. For example, a point in the plane has two degrees of freedom for translation: its two coordinates; a non-infinite ...
, n is the number of observations and x-bar is the sample average. A simple approximation to the exact interval endpoints can be derived using a normal approximation to the distribution. This approximation gives the following values for a 95% confidence interval:
\begin
\lambda_\text &= \widehat\left(1 - \frac\right) \\
\lambda_\text &= \widehat\left(1 + \frac\right)
\end
This approximation may be acceptable for samples containing at least 15 to 20 elements.
Bayesian inference
The conjugate prior
In Bayesian probability theory, if, given a likelihood function
p(x \mid \theta), the posterior distribution p(\theta \mid x) is in the same probability distribution family as the prior probability distribution p(\theta), the prior and posteri ...
for the exponential distribution is the gamma distribution
In probability theory and statistics, the gamma distribution is a versatile two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-squared distribution are special cases of the g ...
(of which the exponential distribution is a special case). The following parameterization of the gamma probability density function is useful:
\operatorname(\lambda; \alpha, \beta) = \frac \lambda^ \exp(-\lambda\beta).
The posterior distribution
The posterior probability is a type of conditional probability that results from updating the prior probability with information summarized by the likelihood via an application of Bayes' rule. From an epistemological perspective, the posterior ...
''p'' can then be expressed in terms of the likelihood function defined above and a gamma prior:
\begin
p(\lambda) &\propto L(\lambda) \Gamma(\lambda; \alpha, \beta) \\
&= \lambda^n \exp\left(-\lambda n\overline\right) \frac \lambda^ \exp(-\lambda \beta) \\
&\propto \lambda^ \exp(-\lambda \left(\beta + n\overline\right)).
\end
Now the posterior density ''p'' has been specified up to a missing normalizing constant. Since it has the form of a gamma pdf, this can easily be filled in, and one obtains:
p(\lambda) = \operatorname(\lambda; \alpha + n, \beta + n\overline).
Here the hyperparameter ''α'' can be interpreted as the number of prior observations, and ''β'' as the sum of the prior observations.
The posterior mean here is:
\frac.
Occurrence and applications
Occurrence of events
The exponential distribution occurs naturally when describing the lengths of the inter-arrival times in a homogeneous Poisson process
In probability theory, statistics and related fields, a Poisson point process (also known as: Poisson random measure, Poisson random point field and Poisson point field) is a type of mathematical object that consists of Point (geometry), points ...
.
The exponential distribution may be viewed as a continuous counterpart of the geometric distribution
In probability theory and statistics, the geometric distribution is either one of two discrete probability distributions:
* The probability distribution of the number X of Bernoulli trials needed to get one success, supported on \mathbb = \;
* T ...
, which describes the number of Bernoulli trial
In the theory of probability and statistics, a Bernoulli trial (or binomial trial) is a random experiment with exactly two possible outcomes, "success" and "failure", in which the probability of success is the same every time the experiment is ...
s necessary for a ''discrete'' process to change state. In contrast, the exponential distribution describes the time for a continuous process to change state.
In real-world scenarios, the assumption of a constant rate (or probability per unit time) is rarely satisfied. For example, the rate of incoming phone calls differs according to the time of day. But if we focus on a time interval during which the rate is roughly constant, such as from 2 to 4 p.m. during work days, the exponential distribution can be used as a good approximate model for the time until the next phone call arrives. Similar caveats apply to the following examples which yield approximately exponentially distributed variables:
* The time until a radioactive particle decay
In particle physics, particle decay is the spontaneous process of one unstable subatomic particle transforming into multiple other particles. The particles created in this process (the ''final state'') must each be less massive than the original ...
s, or the time between clicks of a Geiger counter
A Geiger counter (, ; also known as a Geiger–Müller counter or G-M counter) is an electronic instrument for detecting and measuring ionizing radiation with the use of a Geiger–Müller tube. It is widely used in applications such as radiat ...
* The time between receiving one telephone call and the next
* The time until default (on payment to company debt holders) in reduced-form credit risk modeling
Exponential variables can also be used to model situations where certain events occur with a constant probability per unit length, such as the distance between mutation
In biology, a mutation is an alteration in the nucleic acid sequence of the genome of an organism, virus, or extrachromosomal DNA. Viral genomes contain either DNA or RNA. Mutations result from errors during DNA or viral replication, ...
s on a DNA
Deoxyribonucleic acid (; DNA) is a polymer composed of two polynucleotide chains that coil around each other to form a double helix. The polymer carries genetic instructions for the development, functioning, growth and reproduction of al ...
strand, or between roadkill
Roadkill is a wild animal that has been killed by collision with motor vehicles. Wildlife-vehicle collisions (WVC) have increasingly been the topic of academic research to understand the causes, and how they can be mitigated.
History
Essenti ...
s on a given road.
In queuing theory
Queueing theory is the mathematical study of waiting lines, or queues. A queueing model is constructed so that queue lengths and waiting time can be predicted. Queueing theory is generally considered a branch of operations research because th ...
, the service times of agents in a system (e.g. how long it takes for a bank teller etc. to serve a customer) are often modeled as exponentially distributed variables. (The arrival of customers for instance is also modeled by the Poisson distribution
In probability theory and statistics, the Poisson distribution () is a discrete probability distribution that expresses the probability of a given number of events occurring in a fixed interval of time if these events occur with a known const ...
if the arrivals are independent and distributed identically.) The length of a process that can be thought of as a sequence of several independent tasks follows the Erlang distribution
The Erlang distribution is a two-parameter family of continuous probability distributions with Support (mathematics), support x \in [0, \infty). The two parameters are:
* a positive integer k, the "shape", and
* a positive real number \lambda, ...
(which is the distribution of the sum of several independent exponentially distributed variables).
Reliability theory and reliability engineering also make extensive use of the exponential distribution. Because of the memoryless property of this distribution, it is well-suited to model the constant hazard rate portion of the bathtub curve
The bathtub curve is a particular shape of a failure rate graph. This graph is used in reliability engineering and deterioration modeling. The 'bathtub' refers to the shape of a line that curves up at both ends, similar in shape to a bathtub. Th ...
used in reliability theory. It is also very convenient because it is so easy to add failure rate
Failure is the social concept of not meeting a desirable or intended objective, and is usually viewed as the opposite of success. The criteria for failure depends on context, and may be relative to a particular observer or belief system. On ...
s in a reliability model. The exponential distribution is however not appropriate to model the overall lifetime of organisms or technical devices, because the "failure rates" here are not constant: more failures occur for very young and for very old systems.
In physics
Physics is the scientific study of matter, its Elementary particle, fundamental constituents, its motion and behavior through space and time, and the related entities of energy and force. "Physical science is that department of knowledge whi ...
, if you observe a gas
Gas is a state of matter that has neither a fixed volume nor a fixed shape and is a compressible fluid. A ''pure gas'' is made up of individual atoms (e.g. a noble gas like neon) or molecules of either a single type of atom ( elements such as ...
at a fixed temperature
Temperature is a physical quantity that quantitatively expresses the attribute of hotness or coldness. Temperature is measurement, measured with a thermometer. It reflects the average kinetic energy of the vibrating and colliding atoms making ...
and pressure
Pressure (symbol: ''p'' or ''P'') is the force applied perpendicular to the surface of an object per unit area over which that force is distributed. Gauge pressure (also spelled ''gage'' pressure)The preferred spelling varies by country and eve ...
in a uniform gravitational field
In physics, a gravitational field or gravitational acceleration field is a vector field used to explain the influences that a body extends into the space around itself. A gravitational field is used to explain gravitational phenomena, such as ...
, the heights of the various molecules also follow an approximate exponential distribution, known as the Barometric formula
The barometric formula is a formula used to model how the air pressure (or air density) changes with altitude.
Pressure equations
There are two equations for computing pressure as a function of height. The first equation is applicable to the ...
. This is a consequence of the entropy property mentioned below.
In hydrology
Hydrology () is the scientific study of the movement, distribution, and management of water on Earth and other planets, including the water cycle, water resources, and drainage basin sustainability. A practitioner of hydrology is called a hydro ...
, the exponential distribution is used to analyze extreme values of such variables as monthly and annual maximum values of daily rainfall and river discharge volumes.
:The blue picture illustrates an example of fitting the exponential distribution to ranked annually maximum one-day rainfalls showing also the 90% confidence belt based on the binomial distribution
In probability theory and statistics, the binomial distribution with parameters and is the discrete probability distribution of the number of successes in a sequence of statistical independence, independent experiment (probability theory) ...
. The rainfall data are represented by plotting positions as part of the cumulative frequency analysis.
In operating-rooms management, the distribution of surgery duration for a category of surgeries with no typical work-content (like in an emergency room, encompassing all types of surgeries).
Prediction
Having observed a sample of ''n'' data points from an unknown exponential distribution a common task is to use these samples to make predictions about future data from the same source. A common predictive distribution over future samples is the so-called plug-in distribution, formed by plugging a suitable estimate for the rate parameter ''λ'' into the exponential density function. A common choice of estimate is the one provided by the principle of maximum likelihood, and using this yields the predictive density over a future sample ''x''''n''+1, conditioned on the observed samples ''x'' = (''x''1, ..., ''xn'') given by
p_(x_ \mid x_1, \ldots, x_n) = \left( \frac1 \right) \exp \left( - \frac \right).
The Bayesian approach provides a predictive distribution which takes into account the uncertainty of the estimated parameter, although this may depend crucially on the choice of prior.
A predictive distribution free of the issues of choosing priors that arise under the subjective Bayesian approach is
p_(x_ \mid x_1, \ldots, x_n) = \frac,
which can be considered as
# a frequentist confidence distribution, obtained from the distribution of the pivotal quantity /;
# a profile predictive likelihood, obtained by eliminating the parameter ''λ'' from the joint likelihood of ''x''''n''+1 and ''λ'' by maximization;
# an objective Bayesian predictive posterior distribution, obtained using the non-informative Jeffreys prior
In Bayesian statistics, the Jeffreys prior is a non-informative prior distribution for a parameter space. Named after Sir Harold Jeffreys, its density function is proportional to the square root of the determinant of the Fisher information matri ...
1/''λ'';
# the Conditional Normalized Maximum Likelihood (CNML) predictive distribution, from information theoretic considerations.
The accuracy of a predictive distribution may be measured using the distance or divergence between the true exponential distribution with rate parameter, ''λ''0, and the predictive distribution based on the sample ''x''. The Kullback–Leibler divergence
In mathematical statistics, the Kullback–Leibler (KL) divergence (also called relative entropy and I-divergence), denoted D_\text(P \parallel Q), is a type of statistical distance: a measure of how much a model probability distribution is diff ...
is a commonly used, parameterisation free measure of the difference between two distributions. Letting Δ(''λ''0, , ''p'') denote the Kullback–Leibler divergence between an exponential with rate parameter ''λ''0 and a predictive distribution ''p'' it can be shown that
\begin
\operatorname_ \left \Delta(\lambda_0\parallel p_) \right&= \psi(n) + \frac - \log(n) \\
\operatorname_ \left \Delta(\lambda_0\parallel p_) \right&= \psi(n) + \frac - \log(n)
\end
where the expectation is taken with respect to the exponential distribution with rate parameter , and is the digamma function. It is clear that the CNML predictive distribution is strictly superior to the maximum likelihood plug-in distribution in terms of average Kullback–Leibler divergence for all sample sizes .
Random variate generation
A conceptually very simple method for generating exponential variates is based on inverse transform sampling
Inverse transform sampling (also known as inversion sampling, the inverse probability integral transform, the inverse transformation method, or the Smirnov transform) is a basic method for pseudo-random number sampling, i.e., for generating sampl ...
: Given a random variate ''U'' drawn from the uniform distribution on the unit interval , the variate
T = F^(U)
has an exponential distribution, where ''F'' is the quantile function
In probability and statistics, the quantile function is a function Q: ,1\mapsto \mathbb which maps some probability x \in ,1/math> of a random variable v to the value of the variable y such that P(v\leq y) = x according to its probability distr ...
, defined by
F^(p)=\frac.
Moreover, if ''U'' is uniform on (0, 1), then so is 1 − ''U''. This means one can generate exponential variates as follows:
T = \frac.
Other methods for generating exponential variates are discussed by Knuth[ Donald E. Knuth (1998). '']The Art of Computer Programming
''The Art of Computer Programming'' (''TAOCP'') is a comprehensive multi-volume monograph written by the computer scientist Donald Knuth presenting programming algorithms and their analysis. it consists of published volumes 1, 2, 3, 4A, and 4 ...
'', volume 2: ''Seminumerical Algorithms'', 3rd edn. Boston: Addison–Wesley. . ''See section 3.4.1, p. 133.'' and Devroye.[Luc Devroye (1986). ]
Non-Uniform Random Variate Generation
'. New York: Springer-Verlag. . ''Se
chapter IX
section 2, pp. 392–401.''
A fast method for generating a set of ready-ordered exponential variates without using a sorting routine is also available.[
]
See also
* Dead time – an application of exponential distribution to particle detector analysis.
* Laplace distribution
In probability theory and statistics, the Laplace distribution is a continuous probability distribution named after Pierre-Simon Laplace. It is also sometimes called the double exponential distribution, because it can be thought of as two exponen ...
, or the "double exponential distribution".
* Relationships among probability distributions
* Marshall–Olkin exponential distribution
References
External links
*
Online calculator of Exponential Distribution
{{DEFAULTSORT:Exponential Distribution
Continuous distributions
Exponentials
Poisson point processes
Conjugate prior distributions
Exponential family distributions
Infinitely divisible probability distributions
Survival analysis