In
statistics
Statistics (from German: '' Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, indust ...
, the term linear model is used in different ways according to the context. The most common occurrence is in connection with regression models and the term is often taken as synonymous with
linear regression
In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is cal ...
model. However, the term is also used in
time series analysis with a different meaning. In each case, the designation "linear" is used to identify a subclass of models for which substantial reduction in the complexity of the related
statistical theory is possible.
Linear regression models
For the regression case, the
statistical model
A statistical model is a mathematical model that embodies a set of statistical assumptions concerning the generation of sample data (and similar data from a larger population). A statistical model represents, often in considerably idealized form ...
is as follows. Given a (random) sample
the relation between the observations
and the
independent variables
Dependent and independent variables are variables in mathematical modeling, statistical modeling and experimental sciences. Dependent variables receive this name because, in an experiment, their values are studied under the supposition or deman ...
is formulated as
:
where
may be
nonlinear
In mathematics and science, a nonlinear system is a system in which the change of the output is not proportional to the change of the input. Nonlinear problems are of interest to engineers, biologists, physicists, mathematicians, and many oth ...
functions. In the above, the quantities
are
random variable
A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...
s representing errors in the relationship. The "linear" part of the designation relates to the appearance of the
regression coefficients,
in a linear way in the above relationship. Alternatively, one may say that the predicted values corresponding to the above model, namely
:
are linear functions of the
.
Given that estimation is undertaken on the basis of a
least squares
The method of least squares is a standard approach in regression analysis to approximate the solution of overdetermined systems (sets of equations in which there are more equations than unknowns) by minimizing the sum of the squares of the re ...
analysis, estimates of the unknown parameters
are determined by minimising a sum of squares function
:
From this, it can readily be seen that the "linear" aspect of the model means the following:
:*the function to be minimised is a quadratic function of the
for which minimisation is a relatively simple problem;
:*the derivatives of the function are linear functions of the
making it easy to find the minimising values;
:*the minimising values
are linear functions of the observations
;
:*the minimising values
are linear functions of the random errors
which makes it relatively easy to determine the statistical properties of the estimated values of
.
Time series models
An example of a linear time series model is an
autoregressive moving average model. Here the model for values in a time series can be written in the form
:
where again the quantities
are random variables representing
innovations which are new random effects that appear at a certain time but also affect values of
at later times. In this instance the use of the term "linear model" refers to the structure of the above relationship in representing
as a linear function of past values of the same time series and of current and past values of the innovations.
[Priestley, M.B. (1988) ''Non-linear and Non-stationary time series analysis'', Academic Press. ] This particular aspect of the structure means that it is relatively simple to derive relations for the mean and
covariance
In probability theory and statistics, covariance is a measure of the joint variability of two random variables. If the greater values of one variable mainly correspond with the greater values of the other variable, and the same holds for the le ...
properties of the time series. Note that here the "linear" part of the term "linear model" is not referring to the coefficients
and
, as it would be in the case of a regression model, which looks structurally similar.
Other uses in statistics
There are some other instances where "nonlinear model" is used to contrast with a linearly structured model, although the term "linear model" is not usually applied. One example of this is
nonlinear dimensionality reduction
Nonlinear dimensionality reduction, also known as manifold learning, refers to various related techniques that aim to project high-dimensional data onto lower-dimensional latent manifolds, with the goal of either visualizing the data in the low-d ...
.
See also
*
General linear model
*
Generalized linear model
In statistics, a generalized linear model (GLM) is a flexible generalization of ordinary linear regression. The GLM generalizes linear regression by allowing the linear model to be related to the response variable via a ''link function'' and by ...
*
Linear predictor function In statistics and in machine learning, a linear predictor function is a linear function ( linear combination) of a set of coefficients and explanatory variables (independent variables), whose value is used to predict the outcome of a dependent vari ...
*
Linear system
In systems theory, a linear system is a mathematical model of a system based on the use of a linear operator.
Linear systems typically exhibit features and properties that are much simpler than the nonlinear case.
As a mathematical abstraction ...
*
Linear regression
In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is cal ...
*
Statistical model
A statistical model is a mathematical model that embodies a set of statistical assumptions concerning the generation of sample data (and similar data from a larger population). A statistical model represents, often in considerably idealized form ...
References
{{Authority control
Curve fitting
Regression models
ar:نموذج الانحدار الخطي
fr:Modèle linéaire