In
statistics
Statistics (from German: '' Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, indust ...
, efficiency is a measure of quality of an
estimator, of an
experimental design
The design of experiments (DOE, DOX, or experimental design) is the design of any task that aims to describe and explain the variation of information under conditions that are hypothesized to reflect the variation. The term is generally associ ...
, or of a
hypothesis testing procedure. Essentially, a more efficient estimator, needs fewer input data or observations than a less efficient one to achieve the
Cramér–Rao bound
In estimation theory and statistics, the Cramér–Rao bound (CRB) expresses a lower bound on the variance of unbiased estimators of a deterministic (fixed, though unknown) parameter, the variance of any such estimator is at least as high as the ...
.
An ''efficient estimator'' is characterized by having the smallest possible
variance, indicating that there is a small
deviance between the estimated value and the "true" value in the
L2 norm sense.
The relative efficiency of two procedures is the ratio of their efficiencies, although often this concept is used where the comparison is made between a given procedure and a notional "best possible" procedure. The efficiencies and the relative efficiency of two procedures theoretically depend on the sample size available for the given procedure, but it is often possible to use the asymptotic relative efficiency (defined as the limit of the relative efficiencies as the sample size grows) as the principal comparison measure.
Estimators
The efficiency of an
unbiased estimator, ''T'', of a
parameter
A parameter (), generally, is any characteristic that can help in defining or classifying a particular system (meaning an event, project, object, situation, etc.). That is, a parameter is an element of a system that is useful, or critical, when ...
''θ'' is defined as
:
where
is the
Fisher information of the sample. Thus ''e''(''T'') is the minimum possible variance for an unbiased estimator divided by its actual variance. The
Cramér–Rao bound
In estimation theory and statistics, the Cramér–Rao bound (CRB) expresses a lower bound on the variance of unbiased estimators of a deterministic (fixed, though unknown) parameter, the variance of any such estimator is at least as high as the ...
can be used to prove that ''e''(''T'') ≤ 1.
Efficient estimators
An efficient estimator is an
estimator that estimates the quantity of interest in some “best possible” manner. The notion of “best possible” relies upon the choice of a particular
loss function — the function which quantifies the relative degree of undesirability of estimation errors of different magnitudes. The most common choice of the loss function is
quadratic, resulting in the
mean squared error criterion of optimality.
In general, the spread of an estimator around the parameter θ is a measure of estimator efficiency and performance. This performance can be calculated by finding the mean squared error. More formally, let ''T'' be an estimator for the parameter ''θ''. The mean squared error of ''T'' is the value