In
time series analysis
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. ...
(or forecasting) — as conducted in
statistics,
signal processing
Signal processing is an electrical engineering subfield that focuses on analyzing, modifying and synthesizing '' signals'', such as sound, images, and scientific measurements. Signal processing techniques are used to optimize transmissions, ...
, and many other fields — the innovation is the difference between the observed value of a variable at time ''t'' and the optimal forecast of that value based on information available prior to time ''t''. If the forecasting method is working correctly, successive innovations are uncorrelated with each other, i.e., constitute a
white noise
In signal processing, white noise is a random signal having equal intensity at different frequencies, giving it a constant power spectral density. The term is used, with this or similar meanings, in many scientific and technical disciplines, ...
time series. Thus it can be said that the innovation time series is obtained from the measurement time series by a process of 'whitening', or removing the predictable component. The use of the term innovation in the sense described here is due to
Hendrik Bode and
Claude Shannon
Claude Elwood Shannon (April 30, 1916 – February 24, 2001) was an American mathematician, electrical engineer, and cryptographer known as a "father of information theory".
As a 21-year-old master's degree student at the Massachusetts In ...
(1950) in their discussion of the
Wiener filter
In signal processing, the Wiener filter is a filter used to produce an estimate of a desired or target random process by linear time-invariant ( LTI) filtering of an observed noisy process, assuming known stationary signal and noise spectra, and ...
problem, although the notion was already implicit in the work of
Kolmogorov
Andrey Nikolaevich Kolmogorov ( rus, Андре́й Никола́евич Колмого́ров, p=ɐnˈdrʲej nʲɪkɐˈlajɪvʲɪtɕ kəlmɐˈɡorəf, a=Ru-Andrey Nikolaevich Kolmogorov.ogg, 25 April 1903 – 20 October 1987) was a Sovi ...
.
[Mitter, S. K. (1982). Nonlinear filtering of diffusion processes a guided tour. In ''Advances in Filtering and Optimal Stochastic Control'' (pp. 256-266). Springer, Berlin, Heidelberg.]
In contrast, the residual is the difference between the observed value of a variable at time ''t'' and the optimal updated state of that value based on information available till (including) time ''t''.
See also
*
Kalman filter
For statistics and control theory, Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, and produces estima ...
*
Filtering problem (stochastic processes)
In the theory of stochastic processes, filtering describes the problem of determining the state of a system from an incomplete and potentially noisy set of observations. While originally motivated by problems in engineering, filtering found applic ...
*
Errors and residuals in statistics
In statistics and optimization, errors and residuals are two closely related and easily confused measures of the deviation of an observed value of an element of a statistical sample from its " true value" (not necessarily observable). The err ...
*
Innovation butterfly
References
{{DEFAULTSORT:Innovation (Signal Processing)
Statistical signal processing