Estimator Vs Estimate
   HOME

TheInfoList



Click Here for Items Related To - Estimator Vs Estimate
OR:
Estimator Vs Estimate on:   [Wikipedia]   [Google]   [Amazon]

In
statistics Statistics (from German language, German: ', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a s ...
, an estimator is a rule for calculating an
estimate Estimation (or estimating) is the process of finding an estimate or approximation, which is a value that is usable for some purpose even if input data may be incomplete, uncertain, or unstable. The value is nonetheless usable because it is de ...
of a given
quantity Quantity or amount is a property that can exist as a multitude or magnitude, which illustrate discontinuity and continuity. Quantities can be compared in terms of "more", "less", or "equal", or by assigning a numerical value multiple of a u ...
based on observed data: thus the rule (the estimator), the quantity of interest (the
estimand An estimand is a quantity that is to be estimated in a statistical analysis. The term is used to distinguish the target of inference from the method used to obtain an approximation of this target (i.e., the estimator) and the specific value obtain ...
) and its result (the estimate) are distinguished. For example, the
sample mean The sample mean (sample average) or empirical mean (empirical average), and the sample covariance or empirical covariance are statistics computed from a sample of data on one or more random variables. The sample mean is the average value (or me ...
is a commonly used estimator of the
population mean In statistics, a population is a set of similar items or events which is of interest for some question or experiment. A statistical population can be a group of existing objects (e.g. the set of all stars within the Milky Way galaxy) or a hyp ...
. There are point and interval estimators. The
point estimator In statistics, point estimation involves the use of sample data to calculate a single value (known as a point estimate since it identifies a point in some parameter space) which is to serve as a "best guess" or "best estimate" of an unknown popu ...
s yield single-valued results. This is in contrast to an interval estimator, where the result would be a range of plausible values. "Single value" does not necessarily mean "single number", but includes vector valued or function valued estimators. ''
Estimation theory Estimation theory is a branch of statistics that deals with estimating the values of Statistical parameter, parameters based on measured empirical data that has a random component. The parameters describe an underlying physical setting in such ...
'' is concerned with the properties of estimators; that is, with defining properties that can be used to compare different estimators (different rules for creating estimates) for the same quantity, based on the same data. Such properties can be used to determine the best rules to use under given circumstances. However, in
robust statistics Robust statistics are statistics that maintain their properties even if the underlying distributional assumptions are incorrect. Robust Statistics, statistical methods have been developed for many common problems, such as estimating location parame ...
, statistical theory goes on to consider the balance between having good properties, if tightly defined assumptions hold, and having worse properties that hold under wider conditions.


Background

An "estimator" or "
point estimate In statistics, point estimation involves the use of sample data to calculate a single value (known as a point estimate since it identifies a point in some parameter space) which is to serve as a "best guess" or "best estimate" of an unknown popu ...
" is a
statistic A statistic (singular) or sample statistic is any quantity computed from values in a sample which is considered for a statistical purpose. Statistical purposes include estimating a population parameter, describing a sample, or evaluating a hypot ...
(that is, a function of the data) that is used to infer the value of an unknown
parameter A parameter (), generally, is any characteristic that can help in defining or classifying a particular system (meaning an event, project, object, situation, etc.). That is, a parameter is an element of a system that is useful, or critical, when ...
in a
statistical model A statistical model is a mathematical model that embodies a set of statistical assumptions concerning the generation of Sample (statistics), sample data (and similar data from a larger Statistical population, population). A statistical model repre ...
. A common way of phrasing it is "the estimator is the method selected to obtain an estimate of an unknown parameter". The parameter being estimated is sometimes called the ''
estimand An estimand is a quantity that is to be estimated in a statistical analysis. The term is used to distinguish the target of inference from the method used to obtain an approximation of this target (i.e., the estimator) and the specific value obtain ...
''. It can be either finite-dimensional (in parametric and semi-parametric models), or infinite-dimensional ( semi-parametric and non-parametric models). If the parameter is denoted \theta then the estimator is traditionally written by adding a
circumflex The circumflex () is a diacritic in the Latin and Greek scripts that is also used in the written forms of many languages and in various romanization and transcription schemes. It received its English name from "bent around"a translation of ...
over the symbol: \widehat. Being a function of the data, the estimator is itself a
random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a Mathematics, mathematical formalization of a quantity or object which depends on randomness, random events. The term 'random variable' in its mathema ...
; a particular realization of this random variable is called the "estimate". Sometimes the words "estimator" and "estimate" are used interchangeably. The definition places virtually no restrictions on which functions of the data can be called the "estimators". The attractiveness of different estimators can be judged by looking at their properties, such as
unbiasedness In statistics, the bias of an estimator (or bias function) is the difference between this estimator's expected value and the true value of the parameter being estimated. An estimator or decision rule with zero bias is called ''unbiased''. In stat ...
,
mean square error In statistics, the mean squared error (MSE) or mean squared deviation (MSD) of an estimator (of a procedure for estimating an unobserved quantity) measures the average of the squares of the errors—that is, the average squared difference between ...
,
consistency In deductive logic, a consistent theory is one that does not lead to a logical contradiction. A theory T is consistent if there is no formula \varphi such that both \varphi and its negation \lnot\varphi are elements of the set of consequences ...
,
asymptotic distribution In mathematics and statistics, an asymptotic distribution is a probability distribution that is in a sense the limiting distribution of a sequence of distributions. One of the main uses of the idea of an asymptotic distribution is in providing appr ...
, etc. The construction and comparison of estimators are the subjects of the
estimation theory Estimation theory is a branch of statistics that deals with estimating the values of Statistical parameter, parameters based on measured empirical data that has a random component. The parameters describe an underlying physical setting in such ...
. In the context of
decision theory Decision theory or the theory of rational choice is a branch of probability theory, probability, economics, and analytic philosophy that uses expected utility and probabilities, probability to model how individuals would behave Rationality, ratio ...
, an estimator is a type of
decision rule In decision theory, a decision rule is a function which maps an observation to an appropriate action. Decision rules play an important role in the theory of statistics and economics, and are closely related to the concept of a strategy in game ...
, and its performance may be evaluated through the use of
loss function In mathematical optimization and decision theory, a loss function or cost function (sometimes also called an error function) is a function that maps an event or values of one or more variables onto a real number intuitively representing some "cost ...
s. When the word "estimator" is used without a qualifier, it usually refers to point estimation. The estimate in this case is a single point in the
parameter space The parameter space is the space of all possible parameter values that define a particular mathematical model. It is also sometimes called weight space, and is often a subset of finite-dimensional Euclidean space. In statistics, parameter spaces a ...
. There also exists another type of estimator: interval estimators, where the estimates are subsets of the parameter space. The problem of
density estimation In statistics, probability density estimation or simply density estimation is the construction of an estimate, based on observed data, of an unobservable underlying probability density function. The unobservable density function is thought o ...
arises in two applications. Firstly, in estimating the
probability density function In probability theory, a probability density function (PDF), density function, or density of an absolutely continuous random variable, is a Function (mathematics), function whose value at any given sample (or point) in the sample space (the s ...
s of random variables and secondly in estimating the spectral density function of a
time series In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. ...
. In these problems the estimates are functions that can be thought of as point estimates in an infinite dimensional space, and there are corresponding interval estimation problems.


Definition

Suppose a fixed ''parameter'' \theta needs to be estimated. Then an "estimator" is a function that maps the
sample space In probability theory, the sample space (also called sample description space, possibility space, or outcome space) of an experiment or random trial is the set of all possible outcomes or results of that experiment. A sample space is usually den ...
to a set of ''sample estimates''. An estimator of \theta is usually denoted by the symbol \widehat. It is often convenient to express the theory using the
algebra of random variables In statistics, the algebra of random variables provides rules for the symbolic manipulation of random variables, while avoiding delving too deeply into the mathematically sophisticated ideas of probability theory. Its symbolism allows the treat ...
: thus if ''X'' is used to denote a
random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a Mathematics, mathematical formalization of a quantity or object which depends on randomness, random events. The term 'random variable' in its mathema ...
corresponding to the observed data, the estimator (itself treated as a random variable) is symbolised as a function of that random variable, \widehat(X). The estimate for a particular observed data value x (i.e. for X=x) is then \widehat(x), which is a fixed value. Often an abbreviated notation is used in which \widehat is interpreted directly as a
random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a Mathematics, mathematical formalization of a quantity or object which depends on randomness, random events. The term 'random variable' in its mathema ...
, but this can cause confusion.


Quantified properties

The following definitions and attributes are relevant.


Error

For a given sample x , the "
error An error (from the Latin , meaning 'to wander'Oxford English Dictionary, s.v. “error (n.), Etymology,” September 2023, .) is an inaccurate or incorrect action, thought, or judgement. In statistics, "error" refers to the difference between t ...
" of the estimator \widehat is defined as :e(x)=\widehat(x) - \theta, where \theta is the parameter being estimated. The error, ''e'', depends not only on the estimator (the estimation formula or procedure), but also on the sample.


Mean squared error

The
mean squared error In statistics, the mean squared error (MSE) or mean squared deviation (MSD) of an estimator (of a procedure for estimating an unobserved quantity) measures the average of the squares of the errors—that is, the average squared difference betwee ...
of \widehat is defined as the expected value (probability-weighted average, over all samples) of the squared errors; that is, :\operatorname(\widehat) = \operatorname \widehat(X) - \theta)^2 It is used to indicate how far, on average, the collection of estimates are from the single parameter being estimated. Consider the following analogy. Suppose the parameter is the bull's-eye of a target, the estimator is the process of shooting arrows at the target, and the individual arrows are estimates (samples). Then high MSE means the average distance of the arrows from the bull's eye is high, and low MSE means the average distance from the bull's eye is low. The arrows may or may not be clustered. For example, even if all arrows hit the same point, yet grossly miss the target, the MSE is still relatively large. However, if the MSE is relatively low then the arrows are likely more highly clustered (than highly dispersed) around the target.


Sampling deviation

For a given sample x , the ''sampling deviation'' of the estimator \widehat is defined as :d(x) =\widehat(x) - \operatorname( \widehat(X) ) =\widehat(x) - \operatorname( \widehat ), where \operatorname( \widehat(X) ) is the
expected value In probability theory, the expected value (also called expectation, expectancy, expectation operator, mathematical expectation, mean, expectation value, or first Moment (mathematics), moment) is a generalization of the weighted average. Informa ...
of the estimator. The sampling deviation, ''d'', depends not only on the estimator, but also on the sample.


Variance

The
variance In probability theory and statistics, variance is the expected value of the squared deviation from the mean of a random variable. The standard deviation (SD) is obtained as the square root of the variance. Variance is a measure of dispersion ...
of \widehat is the expected value of the squared sampling deviations; that is, \operatorname(\widehat) = \operatorname \widehat - \operatorname[\widehat ^2">widehat.html" ;"title="\widehat - \operatorname[\widehat">\widehat - \operatorname[\widehat ^2/math>. It is used to indicate how far, on average, the collection of estimates are from the ''expected value'' of the estimates. (Note the difference between MSE and variance.) If the parameter is the bull's-eye of a target, and the arrows are estimates, then a relatively high variance means the arrows are dispersed, and a relatively low variance means the arrows are clustered. Even if the variance is low, the cluster of arrows may still be far off-target, and even if the variance is high, the diffuse collection of arrows may still be unbiased. Finally, even if all arrows grossly miss the target, if they nevertheless all hit the same point, the variance is zero.


Bias

The bias of an estimator">bias Bias is a disproportionate weight ''in favor of'' or ''against'' an idea or thing, usually in a way that is inaccurate, closed-minded, prejudicial, or unfair. Biases can be innate or learned. People may develop biases for or against an individ ...
of \widehat is defined as B(\widehat) = \operatorname(\widehat) - \theta. It is the distance between the average of the collection of estimates, and the single parameter being estimated. The bias of \widehat is a function of the true value of \theta so saying that the bias of \widehat is b means that for every \theta the bias of \widehat is b. There are two kinds of estimators: biased estimators and unbiased estimators. Whether an estimator is biased or not can be identified by the relationship between \operatorname(\widehat) - \theta and 0: * If \operatorname(\widehat) - \theta\neq0, \widehat is biased. * If \operatorname(\widehat) - \theta=0, \widehat is unbiased. The bias is also the expected value of the error, since \operatorname(\widehat) - \theta = \operatorname(\widehat - \theta ) . If the parameter is the bull's eye of a target and the arrows are estimates, then a relatively high absolute value for the bias means the average position of the arrows is off-target, and a relatively low absolute bias means the average position of the arrows is on target. They may be dispersed, or may be clustered. The relationship between bias and variance is analogous to the relationship between accuracy and precision. The estimator \widehat is an estimator bias, unbiased estimator of \theta if and only if B(\widehat) = 0. Bias is a property of the estimator, not of the estimate. Often, people refer to a "biased estimate" or an "unbiased estimate", but they really are talking about an "estimate from a biased estimator", or an "estimate from an unbiased estimator". Also, people often confuse the "error" of a single estimate with the "bias" of an estimator. That the error for one estimate is large, does not mean the estimator is biased. In fact, even if all estimates have astronomical absolute values for their errors, if the expected value of the error is zero, the estimator is unbiased. Also, an estimator's being biased does not preclude the error of an estimate from being zero in a particular instance. The ideal situation is to have an unbiased estimator with low variance, and also try to limit the number of samples where the error is extreme (that is, to have few
outliers In statistics, an outlier is a data point that differs significantly from other observations. An outlier may be due to a variability in the measurement, an indication of novel data, or it may be the result of experimental error; the latter ar ...
). Yet unbiasedness is not essential. Often, if just a little bias is permitted, then an estimator can be found with lower mean squared error and/or fewer outlier sample estimates. An alternative to the version of "unbiased" above, is "median-unbiased", where the
median The median of a set of numbers is the value separating the higher half from the lower half of a Sample (statistics), data sample, a statistical population, population, or a probability distribution. For a data set, it may be thought of as the “ ...
of the distribution of estimates agrees with the true value; thus, in the long run half the estimates will be too low and half too high. While this applies immediately only to scalar-valued estimators, it can be extended to any measure of
central tendency In statistics, a central tendency (or measure of central tendency) is a central or typical value for a probability distribution.Weisberg H.F (1992) ''Central Tendency and Variability'', Sage University Paper Series on Quantitative Applications in ...
of a distribution: see median-unbiased estimators. In a practical problem, \widehat can always have functional relationship with \theta. For example, if a genetic theory states there is a type of leaf (starchy green) that occurs with probability p_1=1/4\cdot(\theta + 2), with 0<\theta<1. Then, for n leaves, the random variable N_1, or the number of starchy green leaves, can be modeled with a Bin(n,p_1) distribution. The number can be used to express the following estimator for \theta: \widehat=4/n\cdot N_1-2. One can show that \widehat is an unbiased estimator for \theta: E widehatE /n\cdot N_1-2/math> =4/n\cdot E _12 =4/n\cdot np_1-2 =4\cdot p_1-2 =4\cdot1/4\cdot(\theta+2)-2 =\theta+2-2 =\theta.


Unbiased

A desired property for estimators is the unbiased trait where an estimator is shown to have no systematic tendency to produce estimates larger or smaller than the true parameter. Additionally, unbiased estimators with smaller variances are preferred over larger variances because it will be closer to the "true" value of the parameter. The unbiased estimator with the smallest variance is known as the
minimum-variance unbiased estimator In statistics a minimum-variance unbiased estimator (MVUE) or uniformly minimum-variance unbiased estimator (UMVUE) is an unbiased estimator that has lower variance than any other unbiased estimator for all possible values of the parameter. For pra ...
(MVUE). To find if your estimator is unbiased it is easy to follow along the equation \operatorname E(\widehat) - \theta=0, \widehat. With estimator ''T'' with and parameter of interest \theta solving the previous equation so it is shown as \operatorname E = \theta the estimator is unbiased. Looking at the figure to the right despite \hat being the only unbiased estimator, if the distributions overlapped and were both centered around \theta then distribution \hat would actually be the preferred unbiased estimator. Expectation When looking at quantities in the interest of expectation for the model distribution there is an unbiased estimator which should satisfy the two equations below. :1. \quad \overline X_n = \frac n :2. \quad \operatorname E\left overline X_n \right= \mu Variance Similarly, when looking at quantities in the interest of variance as the model distribution there is also an unbiased estimator that should satisfy the two equations below. :1. \quad S^2_n = \frac\sum_^n (X_i - \bar)^2 : 2. \quad \operatorname E\left ^2_n\right= \sigma^2 Note we are dividing by ''n'' − 1 because if we divided with ''n'' we would obtain an estimator with a negative bias which would thus produce estimates that are too small for \sigma^2. It should also be mentioned that even though S^2_n is unbiased for \sigma^2 the reverse is not true.


Relationships among the quantities

*The mean squared error, variance, and bias, are related: \operatorname(\widehat) = \operatorname(\widehat\theta) + (B(\widehat))^2, i.e. mean squared error = variance + square of bias. In particular, for an unbiased estimator, the variance equals the mean squared error. *The
standard deviation In statistics, the standard deviation is a measure of the amount of variation of the values of a variable about its Expected value, mean. A low standard Deviation (statistics), deviation indicates that the values tend to be close to the mean ( ...
of an estimator \widehat of \theta (the
square root In mathematics, a square root of a number is a number such that y^2 = x; in other words, a number whose ''square'' (the result of multiplying the number by itself, or y \cdot y) is . For example, 4 and −4 are square roots of 16 because 4 ...
of the variance), or an estimate of the standard deviation of an estimator \widehat of \theta, is called the ''
standard error The standard error (SE) of a statistic (usually an estimator of a parameter, like the average or mean) is the standard deviation of its sampling distribution or an estimate of that standard deviation. In other words, it is the standard deviati ...
'' of \widehat. *The bias-variance tradeoff will be used in model complexity, over-fitting and under-fitting. It is mainly used in the field of
supervised learning In machine learning, supervised learning (SL) is a paradigm where a Statistical model, model is trained using input objects (e.g. a vector of predictor variables) and desired output values (also known as a ''supervisory signal''), which are often ...
and
predictive modelling Predictive modelling uses statistics to Prediction, predict outcomes. Most often the event one wants to predict is in the future, but predictive modelling can be applied to any type of unknown event, regardless of when it occurred. For example, pre ...
to diagnose the performance of algorithms.


Behavioral properties


Consistency

A consistent estimator is an estimator whose sequence of estimates converge in probability to the quantity being estimated as the index (usually the
sample size Sample size determination or estimation is the act of choosing the number of observations or replicates to include in a statistical sample. The sample size is an important feature of any empirical study in which the goal is to make inferences abo ...
) grows without bound. In other words, increasing the sample size increases the probability of the estimator being close to the population parameter. Mathematically, an estimator is a consistent estimator for
parameter A parameter (), generally, is any characteristic that can help in defining or classifying a particular system (meaning an event, project, object, situation, etc.). That is, a parameter is an element of a system that is useful, or critical, when ...
''θ'', if and only if for the sequence of estimates , and for all , no matter how small, we have : \lim_\Pr\left\=1 . The consistency defined above may be called weak consistency. The sequence is ''strongly consistent'', if it converges almost surely to the true value. An estimator that converges to a ''multiple'' of a parameter can be made into a consistent estimator by multiplying the estimator by a
scale factor In affine geometry, uniform scaling (or isotropic scaling) is a linear transformation that enlarges (increases) or shrinks (diminishes) objects by a '' scale factor'' that is the same in all directions ( isotropically). The result of uniform sc ...
, namely the true value divided by the asymptotic value of the estimator. This occurs frequently in estimation of scale parameters by measures of statistical dispersion.


Fisher consistency

An estimator can be considered Fisher consistent as long as the estimator is the same functional of the empirical distribution function as the true distribution function. Following the formula: :\widehat = h(T_n), \theta = h(T_\theta) Where T_n and T_\theta are the
empirical distribution function In statistics, an empirical distribution function ( an empirical cumulative distribution function, eCDF) is the Cumulative distribution function, distribution function associated with the empirical measure of a Sampling (statistics), sample. Th ...
and theoretical distribution function, respectively. An easy example to see if some estimator is Fisher consistent is to check the consistency of mean and variance. For example, to check consistency for the mean \widehat = \bar and to check for variance confirm that \widehat^2 = SSD/n.


Asymptotic normality

An asymptotically normal estimator is a consistent estimator whose distribution around the true parameter ''θ'' approaches a
normal distribution In probability theory and statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is f(x) = \frac ...
with standard deviation shrinking in proportion to 1/\sqrt as the sample size ''n'' grows. Using \xrightarrow to denote
convergence in distribution In probability theory, there exist several different notions of convergence of sequences of random variables, including ''convergence in probability'', ''convergence in distribution'', and ''almost sure convergence''. The different notions of conve ...
, ''tn'' is asymptotically normal if :\sqrt(t_n - \theta) \xrightarrow N(0,V), for some ''V''. In this formulation ''V/n'' can be called the ''asymptotic variance'' of the estimator. However, some authors also call ''V'' the ''asymptotic variance''. Note that convergence will not necessarily have occurred for any finite "n", therefore this value is only an approximation to the true variance of the estimator, while in the limit the asymptotic variance (V/n) is simply zero. To be more specific, the distribution of the estimator ''tn'' converges weakly to a
dirac delta function In mathematical analysis, the Dirac delta function (or distribution), also known as the unit impulse, is a generalized function on the real numbers, whose value is zero everywhere except at zero, and whose integral over the entire real line ...
centered at \theta. The
central limit theorem In probability theory, the central limit theorem (CLT) states that, under appropriate conditions, the Probability distribution, distribution of a normalized version of the sample mean converges to a Normal distribution#Standard normal distributi ...
implies asymptotic normality of the
sample mean The sample mean (sample average) or empirical mean (empirical average), and the sample covariance or empirical covariance are statistics computed from a sample of data on one or more random variables. The sample mean is the average value (or me ...
\bar X as an estimator of the true mean. More generally,
maximum likelihood In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed stati ...
estimators are asymptotically normal under fairly weak regularity conditions — see the asymptotics section of the maximum likelihood article. However, not all estimators are asymptotically normal; the simplest examples are found when the true value of a parameter lies on the boundary of the allowable parameter region.


Efficiency

The efficiency of an estimator is used to estimate the quantity of interest in a "minimum error" manner. In reality, there is not an explicit best estimator; there can only be a better estimator. Whether the efficiency of an estimator is better or not is based on the choice of a particular
loss function In mathematical optimization and decision theory, a loss function or cost function (sometimes also called an error function) is a function that maps an event or values of one or more variables onto a real number intuitively representing some "cost ...
, and it is reflected by two naturally desirable properties of estimators: to be unbiased \operatorname(\widehat) - \theta=0 and have minimal
mean squared error In statistics, the mean squared error (MSE) or mean squared deviation (MSD) of an estimator (of a procedure for estimating an unobserved quantity) measures the average of the squares of the errors—that is, the average squared difference betwee ...
(MSE) \operatorname \widehat - \theta )^2/math>. These cannot in general both be satisfied simultaneously: a biased estimator may have a lower mean squared error than any unbiased estimator (see
estimator bias In statistics, the bias of an estimator (or bias function) is the difference between this estimator's expected value and the true value of the parameter being estimated. An estimator or decision rule with zero bias is called ''unbiased''. In stat ...
). This equation relates the mean squared error with the estimator bias: : \operatorname \widehat - \theta )^2(\operatorname(\widehat) - \theta)^2+\operatorname(\widehat\theta)\ The first term represents the mean squared error; the second term represents the square of the estimator bias; and the third term represents the variance of the estimator. The quality of the estimator can be identified from the comparison between the variance, the square of the estimator bias, or the MSE. The variance of the good estimator (good efficiency) would be smaller than the variance of the bad estimator (bad efficiency). The square of an estimator bias with a good estimator would be smaller than the estimator bias with a bad estimator. The MSE of a good estimator would be smaller than the MSE of the bad estimator. Suppose there are two estimator, \widehat\theta_1 is the good estimator and \widehat\theta_2 is the bad estimator. The above relationship can be expressed by the following formulas. : \operatorname(\widehat\theta_1)<\operatorname(\widehat\theta_2) : , \operatorname(\widehat\theta_1) - \theta, <\left, \operatorname(\widehat\theta_2) - \theta\ : \operatorname(\widehat\theta_1)<\operatorname(\widehat\theta_2) Besides using formula to identify the efficiency of the estimator, it can also be identified through the graph. If an estimator is efficient, in the frequency vs. value graph, there will be a curve with high frequency at the center and low frequency on the two sides. For example: If an estimator is not efficient, the frequency vs. value graph, there will be a relatively more gentle curve. To put it simply, the good estimator has a narrow curve, while the bad estimator has a large curve. Plotting these two curves on one graph with a shared ''y''-axis, the difference becomes more obvious. Among unbiased estimators, there often exists one with the lowest variance, called the minimum variance unbiased estimator ( MVUE). In some cases an unbiased
efficient estimator In statistics, efficiency is a measure of quality of an estimator, of an experimental design, or of a hypothesis testing procedure. Essentially, a more efficient estimator needs fewer input data or observations than a less efficient one to achiev ...
exists, which, in addition to having the lowest variance among unbiased estimators, satisfies the
Cramér–Rao bound In estimation theory and statistics, the Cramér–Rao bound (CRB) relates to estimation of a deterministic (fixed, though unknown) parameter. The result is named in honor of Harald Cramér and Calyampudi Radhakrishna Rao, but has also been d ...
, which is an absolute lower bound on variance for statistics of a variable. Concerning such "best unbiased estimators", see also
Cramér–Rao bound In estimation theory and statistics, the Cramér–Rao bound (CRB) relates to estimation of a deterministic (fixed, though unknown) parameter. The result is named in honor of Harald Cramér and Calyampudi Radhakrishna Rao, but has also been d ...
,
Gauss–Markov theorem In statistics, the Gauss–Markov theorem (or simply Gauss theorem for some authors) states that the ordinary least squares (OLS) estimator has the lowest sampling variance within the class of linear unbiased estimators, if the errors in ...
,
Lehmann–Scheffé theorem In statistics, the Lehmann–Scheffé theorem ties together completeness, sufficiency, uniqueness, and best unbiased estimation. The theorem states that any estimator that is unbiased for a given unknown quantity and that depends on the data only ...
, Rao–Blackwell theorem.


Robustness


See also

*
Best linear unbiased estimator Best or The Best may refer to: People * Best (surname), people with the surname Best * Best (footballer, born 1968), retired Portuguese footballer Companies and organizations * Best & Co., an 1879–1971 clothing chain * Best Lock Corporatio ...
(BLUE) *
Invariant estimator In statistics, the concept of being an invariant estimator is a criterion that can be used to compare the properties of different estimators for the same quantity. It is a way of formalising the idea that an estimator should have certain intuitive ...
*
Kalman filter In statistics and control theory, Kalman filtering (also known as linear quadratic estimation) is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, to produce estimates of unk ...
*
Markov chain Monte Carlo In statistics, Markov chain Monte Carlo (MCMC) is a class of algorithms used to draw samples from a probability distribution. Given a probability distribution, one can construct a Markov chain whose elements' distribution approximates it – that ...
(MCMC) *
Maximum a posteriori An estimation procedure that is often claimed to be part of Bayesian statistics is the maximum a posteriori (MAP) estimate of an unknown quantity, that equals the mode of the posterior density with respect to some reference measure, typically ...
(MAP) * Method of moments,
generalized method of moments In econometrics and statistics, the generalized method of moments (GMM) is a generic method for estimating parameters in statistical models. Usually it is applied in the context of semiparametric models, where the parameter of interest is finite-di ...
*
Minimum mean squared error In statistics and signal processing, a minimum mean square error (MMSE) estimator is an estimation method which minimizes the mean square error (MSE), which is a common measure of estimator quality, of the fitted values of a dependent variable. In ...
(MMSE) *
Particle filter Particle filters, also known as sequential Monte Carlo methods, are a set of Monte Carlo algorithms used to find approximate solutions for filtering problems for nonlinear state-space systems, such as signal processing and Bayesian statistical ...
* Pitman closeness criterion *
Sensitivity and specificity In medicine and statistics, sensitivity and specificity mathematically describe the accuracy of a test that reports the presence or absence of a medical condition. If individuals who have the condition are considered "positive" and those who do ...
* Shrinkage estimator *
Signal processing Signal processing is an electrical engineering subfield that focuses on analyzing, modifying and synthesizing ''signals'', such as audio signal processing, sound, image processing, images, Scalar potential, potential fields, Seismic tomograph ...
*
State observer In control theory, a state observer, state estimator, or Luenberger observer is a system that provides an estimate of the state space (controls), internal state of a given real system, from measurements of the Input/output, input and output of th ...
* Testimator *
Wiener filter In signal processing, the Wiener filter is a filter used to produce an estimate of a desired or target random process by linear time-invariant ( LTI) filtering of an observed noisy process, assuming known stationary signal and noise spectra, a ...
*
Well-behaved statistic Although the term well-behaved statistic often seems to be used in the scientific literature in somewhat the same way as is well-behaved in mathematics (that is, to mean "non- pathological") it can also be assigned precise mathematical meaning, an ...


References


Further reading

* . * . * * * {{Citation , last = Shao , first = Jun , title = Mathematical Statistics , publisher =
Springer Springer or springers may refer to: Publishers * Springer Science+Business Media, aka Springer International Publishing, a worldwide publishing group founded in 1842 in Germany formerly known as Springer-Verlag. ** Springer Nature, a multinationa ...
, year = 1998 , isbn = 0-387-98674-X


External links


Fundamentals on Estimation Theory