HOME

TheInfoList



OR:

In statistics, the residual sum of squares (RSS), also known as the sum of squared estimate of errors (SSE), is the sum of the
squares In Euclidean geometry, a square is a regular quadrilateral, which means that it has four equal sides and four equal angles (90- degree angles, π/2 radian angles, or right angles). It can also be defined as a rectangle with two equal-length a ...
of residuals (deviations predicted from actual empirical values of data). It is a measure of the discrepancy between the data and an estimation model, such as a linear regression. A small RSS indicates a tight fit of the model to the data. It is used as an
optimality criterion In statistics, an optimality criterion provides a measure of the fit of the data to a given hypothesis, to aid in model selection. A model is designated as the "best" of the candidate models if it gives the best value of an objective function mea ...
in parameter selection and
model selection Model selection is the task of selecting a statistical model from a set of candidate models, given data. In the simplest cases, a pre-existing set of data is considered. However, the task can also involve the design of experiments such that the ...
. In general, total sum of squares =
explained sum of squares In statistics, the explained sum of squares (ESS), alternatively known as the model sum of squares or sum of squares due to regression (SSR – not to be confused with the residual sum of squares (RSS) or sum of squares of errors), is a quantity ...
+ residual sum of squares. For a proof of this in the multivariate
ordinary least squares In statistics, ordinary least squares (OLS) is a type of linear least squares method for choosing the unknown parameters in a linear regression model (with fixed level-one effects of a linear function of a set of explanatory variables) by the ...
(OLS) case, see partitioning in the general OLS model.


One explanatory variable

In a model with a single explanatory variable, RSS is given by: :\operatorname = \sum_^n (y_i - f(x_i))^2 where ''y''''i'' is the ''i''th value of the variable to be predicted, ''x''''i'' is the ''i''th value of the explanatory variable, and f(x_i) is the predicted value of ''y''''i'' (also termed \hat). In a standard linear simple regression model, y_i = \alpha + \beta x_i+\varepsilon_i\,, where \alpha and \beta are coefficients, ''y'' and ''x'' are the regressand and the regressor, respectively, and ε is the error term. The sum of squares of residuals is the sum of squares of \widehat_i; that is :\operatorname = \sum_^n (\widehat_i)^2 = \sum_^n (y_i - (\widehat + \widehat x_i))^2 where \widehat is the estimated value of the constant term \alpha and \widehat is the estimated value of the slope coefficient \beta.


Matrix expression for the OLS residual sum of squares

The general regression model with observations and explanators, the first of which is a constant unit vector whose coefficient is the regression intercept, is : y = X \beta + e where is an ''n'' × 1 vector of dependent variable observations, each column of the ''n'' × ''k'' matrix is a vector of observations on one of the ''k'' explanators, \beta is a ''k'' × 1 vector of true coefficients, and is an ''n''× 1 vector of the true underlying errors. The
ordinary least squares In statistics, ordinary least squares (OLS) is a type of linear least squares method for choosing the unknown parameters in a linear regression model (with fixed level-one effects of a linear function of a set of explanatory variables) by the ...
estimator for \beta is : X \hat \beta = y \iff : X^\operatorname X \hat \beta = X^\operatorname y \iff : \hat \beta = (X^\operatorname X)^X^\operatorname y. The residual vector \hat e = y - X \hat \beta = y - X (X^\operatorname X)^X^\operatorname y; so the residual sum of squares is: :\operatorname = \hat e ^\operatorname \hat e = \, \hat e \, ^2 , (equivalent to the square of the norm of residuals). In full: :\operatorname = y^\operatorname y - y^\operatorname X(X^\operatorname X)^ X^\operatorname y = y^\operatorname - X(X^\operatorname X)^ X^\operatornamey = y^\operatorname - Hy, where is the hat matrix, or the projection matrix in linear regression.


Relation with Pearson's product-moment correlation

The least-squares regression line is given by :y=ax+b, where b=\bar-a\bar and a=\frac, where S_=\sum_^n(\bar-x_i)(\bar-y_i) and S_=\sum_^n(\bar-x_i)^2. Therefore, : \begin \operatorname & = \sum_^n (y_i - f(x_i))^2= \sum_^n (y_i - (ax_i+b))^2= \sum_^n (y_i - ax_i-\bar + a\bar)^2 \\ pt& = \sum_^n (a(\bar-x_i)-(\bar-y_i))^2=a^2S_-2aS_+S_=S_-aS_=S_ \left(1-\frac \right) \end where S_=\sum_^n (\bar-y_i)^2 . The Pearson product-moment correlation is given by r=\frac; therefore, \operatorname=S_(1-r^2).


See also

* Akaike information criterion#Comparison with least squares * Chi-squared distribution#Applications * Degrees of freedom (statistics)#Sum of squares and degrees of freedom *
Errors and residuals in statistics In statistics and optimization, errors and residuals are two closely related and easily confused measures of the deviation of an observed value of an element of a statistical sample from its "true value" (not necessarily observable). The er ...
* Lack-of-fit sum of squares *
Mean squared error In statistics, the mean squared error (MSE) or mean squared deviation (MSD) of an estimator (of a procedure for estimating an unobserved quantity) measures the average of the squares of the errors—that is, the average squared difference between ...
*
Reduced chi-squared statistic In statistics, the reduced chi-square statistic is used extensively in goodness of fit testing. It is also known as mean squared weighted deviation (MSWD) in isotopic dating and variance of unit weight in the context of weighted least squares. ...
, RSS per degree of freedom *
Squared deviations Squared deviations from the mean (SDM) result from squaring deviations. In probability theory and statistics, the definition of ''variance'' is either the expected value of the SDM (when considering a theoretical distribution) or its average valu ...
* Sum of squares (statistics)


References

* {{cite book , title = Applied Regression Analysis , edition = 3rd , last1= Draper , first1=N.R. , last2=Smith , first2=H. , publisher = John Wiley , year = 1998 , isbn = 0-471-17082-8 Least squares Errors and residuals