HOME

TheInfoList



OR:

Estimation theory is a branch of
statistics Statistics (from German: ''Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, indust ...
that deals with estimating the values of
parameters A parameter (), generally, is any characteristic that can help in defining or classifying a particular system (meaning an event, project, object, situation, etc.). That is, a parameter is an element of a system that is useful, or critical, when ...
based on measured empirical data that has a random component. The parameters describe an underlying physical setting in such a way that their value affects the distribution of the measured data. An ''
estimator In statistics, an estimator is a rule for calculating an estimate of a given quantity based on observed data: thus the rule (the estimator), the quantity of interest (the estimand) and its result (the estimate) are distinguished. For example, th ...
'' attempts to approximate the unknown parameters using the measurements. In estimation theory, two approaches are generally considered: * The probabilistic approach (described in this article) assumes that the measured data is random with
probability distribution In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomeno ...
dependent on the parameters of interest * The set-membership approach assumes that the measured data vector belongs to a set which depends on the parameter vector.


Examples

For example, it is desired to estimate the proportion of a population of voters who will vote for a particular candidate. That proportion is the parameter sought; the estimate is based on a small random sample of voters. Alternatively, it is desired to estimate the probability of a voter voting for a particular candidate, based on some demographic features, such as age. Or, for example, in
radar Radar is a detection system that uses radio waves to determine the distance (''ranging''), angle, and radial velocity of objects relative to the site. It can be used to detect aircraft, ships, spacecraft, guided missiles, motor vehicles, ...
the aim is to find the range of objects (airplanes, boats, etc.) by analyzing the two-way transit timing of received echoes of transmitted pulses. Since the reflected pulses are unavoidably embedded in electrical noise, their measured values are randomly distributed, so that the transit time must be estimated. As another example, in electrical communication theory, the measurements which contain information regarding the parameters of interest are often associated with a noisy
signal In signal processing, a signal is a function that conveys information about a phenomenon. Any quantity that can vary over space or time can be used as a signal to share messages between observers. The '' IEEE Transactions on Signal Processing' ...
.


Basics

For a given model, several statistical "ingredients" are needed so the estimator can be implemented. The first is a
statistical sample In statistics, quality assurance, and survey methodology, sampling is the selection of a subset (a statistical sample) of individuals from within a statistical population to estimate characteristics of the whole population. Statisticians attemp ...
– a set of data points taken from a
random vector In probability, and statistics, a multivariate random variable or random vector is a list of mathematical variables each of whose value is unknown, either because the value has not yet occurred or because there is imperfect knowledge of its val ...
(RV) of size ''N''. Put into a
vector Vector most often refers to: *Euclidean vector, a quantity with a magnitude and a direction *Vector (epidemiology), an agent that carries and transmits an infectious pathogen into another living organism Vector may also refer to: Mathematic ...
, : \mathbf = \begin x \\ x \\ \vdots \\ x -1\end. Secondly, there are ''M'' parameters : \mathbf = \begin \theta_1 \\ \theta_2 \\ \vdots \\ \theta_M \end, whose values are to be estimated. Third, the continuous
probability density function In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) can ...
(pdf) or its discrete counterpart, the
probability mass function In probability and statistics, a probability mass function is a function that gives the probability that a discrete random variable is exactly equal to some value. Sometimes it is also known as the discrete density function. The probability ma ...
(pmf), of the underlying distribution that generated the data must be stated conditional on the values of the parameters: : p(\mathbf , \mathbf).\, It is also possible for the parameters themselves to have a probability distribution (e.g.,
Bayesian statistics Bayesian statistics is a theory in the field of statistics based on the Bayesian interpretation of probability where probability expresses a ''degree of belief'' in an event. The degree of belief may be based on prior knowledge about the event ...
). It is then necessary to define the
Bayesian probability Bayesian probability is an interpretation of the concept of probability, in which, instead of frequency or propensity of some phenomenon, probability is interpreted as reasonable expectation representing a state of knowledge or as quantification ...
: \pi( \mathbf).\, After the model is formed, the goal is to estimate the parameters, with the estimates commonly denoted \hat, where the "hat" indicates the estimate. One common estimator is the
minimum mean squared error In statistics and signal processing, a minimum mean square error (MMSE) estimator is an estimation method which minimizes the mean square error (MSE), which is a common measure of estimator quality, of the fitted values of a dependent variable. In ...
(MMSE) estimator, which utilizes the error between the estimated parameters and the actual value of the parameters : \mathbf = \hat - \mathbf as the basis for optimality. This error term is then squared and the
expected value In probability theory, the expected value (also called expectation, expectancy, mathematical expectation, mean, average, or first moment) is a generalization of the weighted average. Informally, the expected value is the arithmetic mean of a ...
of this squared value is minimized for the MMSE estimator.


Estimators

Commonly used estimators (estimation methods) and topics related to them include: *
Maximum likelihood In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed statis ...
estimators *
Bayes estimator In estimation theory and decision theory, a Bayes estimator or a Bayes action is an estimator or decision rule that minimizes the posterior expected value of a loss function (i.e., the posterior expected loss). Equivalently, it maximizes the p ...
s * Method of moments estimators *
Cramér–Rao bound In estimation theory and statistics, the Cramér–Rao bound (CRB) expresses a lower bound on the variance of unbiased estimators of a deterministic (fixed, though unknown) parameter, the variance of any such estimator is at least as high as the in ...
*
Least squares The method of least squares is a standard approach in regression analysis to approximate the solution of overdetermined systems (sets of equations in which there are more equations than unknowns) by minimizing the sum of the squares of the r ...
*
Minimum mean squared error In statistics and signal processing, a minimum mean square error (MMSE) estimator is an estimation method which minimizes the mean square error (MSE), which is a common measure of estimator quality, of the fitted values of a dependent variable. In ...
(MMSE), also known as Bayes least squared error (BLSE) *
Maximum a posteriori In Bayesian statistics, a maximum a posteriori probability (MAP) estimate is an estimate of an unknown quantity, that equals the mode of the posterior distribution. The MAP can be used to obtain a point estimate of an unobserved quantity on the ...
(MAP) *
Minimum variance unbiased estimator In statistics a minimum-variance unbiased estimator (MVUE) or uniformly minimum-variance unbiased estimator (UMVUE) is an unbiased estimator that has lower variance than any other unbiased estimator for all possible values of the parameter. For pr ...
(MVUE) *
Nonlinear system identification System identification is a method of identifying or measuring the mathematical model of a system from measurements of the system inputs and outputs. The applications of system identification include any system where the inputs and outputs can be mea ...
*
Best linear unbiased estimator Best or The Best may refer to: People * Best (surname), people with the surname Best * Best (footballer, born 1968), retired Portuguese footballer Companies and organizations * Best & Co., an 1879–1971 clothing chain * Best Lock Corporatio ...
(BLUE) *Unbiased estimators — see
estimator bias In statistics, the bias of an estimator (or bias function) is the difference between this estimator's expected value and the true value of the parameter being estimated. An estimator or decision rule with zero bias is called ''unbiased''. In sta ...
. *
Particle filter Particle filters, or sequential Monte Carlo methods, are a set of Monte Carlo algorithms used to solve filtering problems arising in signal processing and Bayesian statistical inference. The filtering problem consists of estimating the inte ...
*
Markov chain Monte Carlo In statistics, Markov chain Monte Carlo (MCMC) methods comprise a class of algorithms for sampling from a probability distribution. By constructing a Markov chain that has the desired distribution as its equilibrium distribution, one can obtain ...
(MCMC) *
Kalman filter For statistics and control theory, Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, and produces esti ...
, and its various derivatives *
Wiener filter In signal processing, the Wiener filter is a filter used to produce an estimate of a desired or target random process by linear time-invariant ( LTI) filtering of an observed noisy process, assuming known stationary signal and noise spectra, and ...


Examples


Unknown constant in additive white Gaussian noise

Consider a received
discrete signal In mathematical dynamics, discrete time and continuous time are two alternative frameworks within which variables that evolve over time are modeled. Discrete time Discrete time views values of variables as occurring at distinct, separate "po ...
, x /math>, of N
independent Independent or Independents may refer to: Arts, entertainment, and media Artist groups * Independents (artist group), a group of modernist painters based in the New Hope, Pennsylvania, area of the United States during the early 1930s * Independ ...
samples that consists of an unknown constant A with
additive white Gaussian noise Additive white Gaussian noise (AWGN) is a basic noise model used in information theory to mimic the effect of many random processes that occur in nature. The modifiers denote specific characteristics: * ''Additive'' because it is added to any nois ...
(AWGN) w /math> with zero
mean There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value (magnitude and sign) of a given data set. For a data set, the ''arith ...
and known
variance In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbe ...
\sigma^2 (''i.e.'', \mathcal(0, \sigma^2)). Since the variance is known then the only unknown parameter is A. The model for the signal is then : x = A + w \quad n=0, 1, \dots, N-1 Two possible (of many) estimators for the parameter A are: * \hat_1 = x /math> * \hat_2 = \frac \sum_^ x /math> which is the
sample mean The sample mean (or "empirical mean") and the sample covariance are statistics computed from a sample of data on one or more random variables. The sample mean is the average value (or mean value) of a sample of numbers taken from a larger pop ...
Both of these estimators have a
mean There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value (magnitude and sign) of a given data set. For a data set, the ''arith ...
of A, which can be shown through taking the
expected value In probability theory, the expected value (also called expectation, expectancy, mathematical expectation, mean, average, or first moment) is a generalization of the weighted average. Informally, the expected value is the arithmetic mean of a ...
of each estimator :\mathrm\left hat_1\right= \mathrm\left \right.html" ;"title="x \right">x \right= A and : \mathrm\left \hat_2 \right= \mathrm\left \right.html" ;"title="\frac \sum_^ x \right">\frac \sum_^ x \right= \frac \left[ \sum_^ \mathrm\left \right.html" ;"title="x \right">x \right\right] = \frac \left[ N A \right] = A At this point, these two estimators would appear to perform the same. However, the difference between them becomes apparent when comparing the variances. :\mathrm \left( \hat_1 \right) = \mathrm \left( x \right) = \sigma^2 and : \mathrm \left( \hat_2 \right) = \mathrm \left( \frac \sum_^ x \right) \overset \frac \left \sum_^_\mathrm_(x[n_\right.html" ;"title=".html" ;"title="\sum_^ \mathrm (x[n">\sum_^ \mathrm (x[n \right">.html" ;"title="\sum_^ \mathrm (x[n">\sum_^ \mathrm (x[n \right= \frac \left[ N \sigma^2 \right] = \frac It would seem that the sample mean is a better estimator since its variance is lower for every ''N'' > 1.


Maximum likelihood

Continuing the example using the
maximum likelihood In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed statis ...
estimator, the
probability density function In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) can ...
(pdf) of the noise for one sample w /math> is :p(w = \frac \exp\left(- \frac w 2 \right) and the probability of x /math> becomes (x /math> can be thought of a \mathcal(A, \sigma^2)) :p(x A) = \frac \exp\left(- \frac (x - A)^2 \right) By
independence Independence is a condition of a person, nation, country, or state in which residents and population, or some portion thereof, exercise self-government, and usually sovereignty, over its territory. The opposite of independence is the st ...
, the probability of \mathbf becomes : p(\mathbf; A) = \prod_^ p(x A) = \frac \exp\left(- \frac \sum_^(x - A)^2 \right) Taking the
natural logarithm The natural logarithm of a number is its logarithm to the base of the mathematical constant , which is an irrational and transcendental number approximately equal to . The natural logarithm of is generally written as , , or sometimes, if ...
of the pdf : \ln p(\mathbf; A) = -N \ln \left(\sigma \sqrt\right) - \frac \sum_^(x - A)^2 and the maximum likelihood estimator is :\hat = \arg \max \ln p(\mathbf; A) Taking the first
derivative In mathematics, the derivative of a function of a real variable measures the sensitivity to change of the function value (output value) with respect to a change in its argument (input value). Derivatives are a fundamental tool of calculus. ...
of the log-likelihood function : \frac \ln p(\mathbf; A) = \frac \left -_A)_\right.html" ;"title="\sum_^(x - A) \right">\sum_^(x - A) \right= \frac \left -_N_A_\right.html" ;"title="\sum_^x - N A \right">\sum_^x - N A \right and setting it to zero : 0 = \frac \left -_N_A_\right.html" ;"title="\sum_^x - N A \right">\sum_^x - N A \right= \sum_^x - N A This results in the maximum likelihood estimator : \hat = \frac \sum_^x which is simply the sample mean. From this example, it was found that the sample mean is the maximum likelihood estimator for N samples of a fixed, unknown parameter corrupted by AWGN.


Cramér–Rao lower bound

To find the Cramér–Rao lower bound (CRLB) of the sample mean estimator, it is first necessary to find the
Fisher information In mathematical statistics, the Fisher information (sometimes simply called information) is a way of measuring the amount of information that an observable random variable ''X'' carries about an unknown parameter ''θ'' of a distribution that mode ...
number : \mathcal(A) = \mathrm \left( \left \frac \ln p(\mathbf; A) \right2 \right) = -\mathrm \left \frac \ln p(\mathbf; A) \right and copying from above : \frac \ln p(\mathbf; A) = \frac \left -_N_A_\right.html" ;"title="\sum_^x - N A \right">\sum_^x - N A \right Taking the second derivative : \frac \ln p(\mathbf; A) = \frac (- N) = \frac and finding the negative expected value is trivial since it is now a deterministic constant -\mathrm \left \frac \ln p(\mathbf; A) \right= \frac Finally, putting the Fisher information into : \mathrm\left( \hat \right) \geq \frac results in : \mathrm\left( \hat \right) \geq \frac Comparing this to the variance of the sample mean (determined previously) shows that the sample mean is ''equal to'' the Cramér–Rao lower bound for all values of N and A. In other words, the sample mean is the (necessarily unique)
efficient estimator In statistics, efficiency is a measure of quality of an estimator, of an experimental design, or of a hypothesis testing procedure. Essentially, a more efficient estimator, needs fewer input data or observations than a less efficient one to achi ...
, and thus also the
minimum variance unbiased estimator In statistics a minimum-variance unbiased estimator (MVUE) or uniformly minimum-variance unbiased estimator (UMVUE) is an unbiased estimator that has lower variance than any other unbiased estimator for all possible values of the parameter. For pr ...
(MVUE), in addition to being the
maximum likelihood In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed statis ...
estimator.


Maximum of a uniform distribution

One of the simplest non-trivial examples of estimation is the estimation of the maximum of a uniform distribution. It is used as a hands-on classroom exercise and to illustrate basic principles of estimation theory. Further, in the case of estimation based on a single sample, it demonstrates philosophical issues and possible misunderstandings in the use of
maximum likelihood In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed statis ...
estimators and likelihood functions. Given a
discrete uniform distribution In probability theory and statistics, the discrete uniform distribution is a symmetric probability distribution wherein a finite number of values are equally likely to be observed; every one of ''n'' values has equal probability 1/''n''. Anothe ...
1,2,\dots,N with unknown maximum, the
UMVU In statistics a minimum-variance unbiased estimator (MVUE) or uniformly minimum-variance unbiased estimator (UMVUE) is an unbiased estimator that has lower variance than any other unbiased estimator for all possible values of the parameter. For pr ...
estimator for the maximum is given by :\frac m - 1 = m + \frac - 1 where ''m'' is the
sample maximum In statistics, the sample maximum and sample minimum, also called the largest observation and smallest observation, are the values of the greatest and least elements of a sample. They are basic summary statistics, used in descriptive statistics ...
and ''k'' is the
sample size Sample size determination is the act of choosing the number of observations or replicates to include in a statistical sample. The sample size is an important feature of any empirical study in which the goal is to make inferences about a population ...
, sampling without replacement. This problem is commonly known as the
German tank problem In the statistical theory of estimation, the German tank problem consists of estimating the maximum of a discrete uniform distribution from sampling without replacement. In simple terms, suppose there exists an unknown number of items which ar ...
, due to application of maximum estimation to estimates of German tank production during
World War II World War II or the Second World War, often abbreviated as WWII or WW2, was a world war that lasted from 1939 to 1945. It involved the vast majority of the world's countries—including all of the great powers—forming two opposing ...
. The formula may be understood intuitively as; :"The sample maximum plus the average gap between observations in the sample", the gap being added to compensate for the negative bias of the sample maximum as an estimator for the population maximum. This has a variance of :\frac\frac \approx \frac \text k \ll N so a standard deviation of approximately N/k, the (population) average size of a gap between samples; compare \frac above. This can be seen as a very simple case of
maximum spacing estimation In statistics, maximum spacing estimation (MSE or MSP), or maximum product of spacing estimation (MPS), is a method for estimating the parameters of a univariate statistical model. The method requires maximization of the geometric mean of ''spaci ...
. The sample maximum is the
maximum likelihood In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed statis ...
estimator for the population maximum, but, as discussed above, it is biased.


Applications

Numerous fields require the use of estimation theory. Some of these fields include: * Interpretation of scientific
experiment An experiment is a procedure carried out to support or refute a hypothesis, or determine the efficacy or likelihood of something previously untried. Experiments provide insight into cause-and-effect by demonstrating what outcome occurs whe ...
s *
Signal processing Signal processing is an electrical engineering subfield that focuses on analyzing, modifying and synthesizing ''signals'', such as sound, images, and scientific measurements. Signal processing techniques are used to optimize transmissions, ...
*
Clinical trial Clinical trials are prospective biomedical or behavioral research studies on human participants designed to answer specific questions about biomedical or behavioral interventions, including new treatments (such as novel vaccines, drugs, dieta ...
s *
Opinion poll An opinion poll, often simply referred to as a survey or a poll (although strictly a poll is an actual election) is a human research survey of public opinion from a particular sample. Opinion polls are usually designed to represent the opinion ...
s *
Quality control Quality control (QC) is a process by which entities review the quality of all factors involved in production. ISO 9000 defines quality control as "a part of quality management focused on fulfilling quality requirements". This approach places ...
*
Telecommunication Telecommunication is the transmission of information by various types of technologies over wire, radio, optical, or other electromagnetic systems. It has its origin in the desire of humans for communication over a distance greater than that ...
s *
Project management Project management is the process of leading the work of a team to achieve all project goals within the given constraints. This information is usually described in project documentation, created at the beginning of the development process. T ...
*
Software engineering Software engineering is a systematic engineering approach to software development. A software engineer is a person who applies the principles of software engineering to design, develop, maintain, test, and evaluate computer software. The term '' ...
*
Control theory Control theory is a field of mathematics that deals with the control of dynamical systems in engineered processes and machines. The objective is to develop a model or algorithm governing the application of system inputs to drive the system to a ...
(in particular
Adaptive control Adaptive control is the control method used by a controller which must adapt to a controlled system with parameters which vary, or are initially uncertain. For example, as an aircraft flies, its mass will slowly decrease as a result of fuel consumpt ...
) * Network intrusion detection system *
Orbit determination Orbit determination is the estimation of orbits of objects such as moons, planets, and spacecraft. One major application is to allow tracking newly observed asteroids and verify that they have not been previously discovered. The basic methods wer ...
Measured data are likely to be subject to
noise Noise is unwanted sound considered unpleasant, loud or disruptive to hearing. From a physics standpoint, there is no distinction between noise and desired sound, as both are vibrations through a medium, such as air or water. The difference ari ...
or uncertainty and it is through statistical
probability Probability is the branch of mathematics concerning numerical descriptions of how likely an event is to occur, or how likely it is that a proposition is true. The probability of an event is a number between 0 and 1, where, roughly speaking, ...
that optimal solutions are sought to extract as much
information Information is an abstract concept that refers to that which has the power to inform. At the most fundamental level information pertains to the interpretation of that which may be sensed. Any natural process that is not completely random, ...
from the data as possible.


See also

*
Best linear unbiased estimator Best or The Best may refer to: People * Best (surname), people with the surname Best * Best (footballer, born 1968), retired Portuguese footballer Companies and organizations * Best & Co., an 1879–1971 clothing chain * Best Lock Corporatio ...
(BLUE) * Completeness (statistics) *
Detection theory Detection theory or signal detection theory is a means to measure the ability to differentiate between information-bearing patterns (called stimulus in living organisms, signal in machines) and random patterns that distract from the information (c ...
*
Efficiency (statistics) In statistics, efficiency is a measure of quality of an estimator, of an experimental design, or of a hypothesis testing procedure. Essentially, a more efficient estimator, needs fewer input data or observations than a less efficient one to achi ...
* Expectation-maximization algorithm (EM algorithm) *
Fermi problem In physics or engineering education, a Fermi problem, Fermi quiz, Fermi question, Fermi estimate, order-of-magnitude problem, order-of-magnitude estimate, or order estimation is an estimation problem designed to teach dimensional analysis or app ...
*
Grey box model In mathematics, statistics, and computational modelling, a grey box modelKroll, Andreas (2000). Grey-box models: Concepts and application. In: New Frontiers in Computational Intelligence and its Applications, vol.57 of Frontiers in artificial in ...
*
Information theory Information theory is the scientific study of the quantification, storage, and communication of information. The field was originally established by the works of Harry Nyquist and Ralph Hartley, in the 1920s, and Claude Shannon in the 1940 ...
*
Least-squares spectral analysis Least-squares spectral analysis (LSSA) is a method of estimating a frequency spectrum, based on a least squares fit of sinusoids to data samples, similar to Fourier analysis. Fourier analysis, the most used spectral method in science, generally ...
*
Matched filter In signal processing, a matched filter is obtained by correlating a known delayed signal, or ''template'', with an unknown signal to detect the presence of the template in the unknown signal. This is equivalent to convolving the unknown signal wi ...
* Maximum entropy spectral estimation *
Nuisance parameter Nuisance (from archaic ''nocence'', through Fr. ''noisance'', ''nuisance'', from Lat. ''nocere'', "to hurt") is a common law tort. It means that which causes offence, annoyance, trouble or injury. A nuisance can be either public (also "common" ...
*
Parametric equation In mathematics, a parametric equation defines a group of quantities as functions of one or more independent variables called parameters. Parametric equations are commonly used to express the coordinates of the points that make up a geometric ob ...
*
Pareto principle The Pareto principle states that for many outcomes, roughly 80% of consequences come from 20% of causes (the "vital few"). Other names for this principle are the 80/20 rule, the law of the vital few, or the principle of factor sparsity. Manage ...
* Rule of three (statistics) * State estimator *
Statistical signal processing Signal processing is an electrical engineering subfield that focuses on analyzing, modifying and synthesizing ''signals'', such as sound, images, and scientific measurements. Signal processing techniques are used to optimize transmissions, ...
*
Sufficiency (statistics) In statistics, a statistic is ''sufficient'' with respect to a statistical model and its associated unknown parameter if "no other statistic that can be calculated from the same sample provides any additional information as to the value of the par ...


Notes


References


Citations


Sources

* ''Theory of Point Estimation'' by E.L. Lehmann and G. Casella. () * ''Systems Cost Engineering'' by Dale Shermon. () * ''Mathematical Statistics and Data Analysis'' by John Rice. () * ''Fundamentals of Statistical Signal Processing: Estimation Theory'' by Steven M. Kay () * ''An Introduction to Signal Detection and Estimation'' by H. Vincent Poor () * ''Detection, Estimation, and Modulation Theory, Part 1'' by Harry L. Van Trees (
website
* ''Optimal State Estimation: Kalman, H-infinity, and Nonlinear Approaches'' by Dan Simo
website
* Ali H. Sayed, Adaptive Filters, Wiley, NJ, 2008, . * Ali H. Sayed, Fundamentals of Adaptive Filtering, Wiley, NJ, 2003, . *
Thomas Kailath Thomas Kailath (born June 7, 1935) is an electrical engineer, information theorist, control engineer, entrepreneur and the Hitachi America Professor of Engineering, Emeritus, at Stanford University. Professor Kailath has authored several book ...
, Ali H. Sayed, and Babak Hassibi, Linear Estimation, Prentice-Hall, NJ, 2000, . * Babak Hassibi, Ali H. Sayed, and
Thomas Kailath Thomas Kailath (born June 7, 1935) is an electrical engineer, information theorist, control engineer, entrepreneur and the Hitachi America Professor of Engineering, Emeritus, at Stanford University. Professor Kailath has authored several book ...
, Indefinite Quadratic Estimation and Control: A Unified Approach to H2 and H\infty Theories, Society for Industrial & Applied Mathematics (SIAM), PA, 1999, . * V.G.Voinov, M.S.Nikulin, "Unbiased estimators and their applications. Vol.1: Univariate case", Kluwer Academic Publishers, 1993, . * V.G.Voinov, M.S.Nikulin, "Unbiased estimators and their applications. Vol.2: Multivariate case", Kluwer Academic Publishers, 1996, .


External links

* {{DSP Signal processing Mathematical and quantitative methods (economics)