In
statistics
Statistics (from German language, German: ', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a s ...
, an additive model (AM) is a
nonparametric regression method. It was suggested by
Jerome H. Friedman and Werner Stuetzle (1981) and is an essential part of the
ACE algorithm. The ''AM'' uses a one-dimensional
smoother to build a restricted class of nonparametric regression models. Because of this, it is less affected by the
curse of dimensionality than a ''p''-dimensional smoother. Furthermore, the ''AM'' is more flexible than a
standard linear model, while being more interpretable than a general regression surface at the cost of approximation errors. Problems with ''AM'', like many other machine-learning methods, include
model selection,
overfitting
In mathematical modeling, overfitting is "the production of an analysis that corresponds too closely or exactly to a particular set of data, and may therefore fail to fit to additional data or predict future observations reliably". An overfi ...
, and
multicollinearity.
Description
Given a
data
Data ( , ) are a collection of discrete or continuous values that convey information, describing the quantity, quality, fact, statistics, other basic units of meaning, or simply sequences of symbols that may be further interpreted for ...
set
of ''n''
statistical units, where
represent predictors and
is the outcome, the ''additive model'' takes the form
:
or
:
Where
,
and
. The functions
are unknown
smooth function
In mathematical analysis, the smoothness of a function is a property measured by the number of continuous derivatives (''differentiability class)'' it has over its domain.
A function of class C^k is a function of smoothness at least ; t ...
s fit from the data. Fitting the ''AM'' (i.e. the functions
) can be done using the
backfitting algorithm proposed by Andreas Buja,
Trevor Hastie and
Robert Tibshirani (1989).
[Buja, A., Hastie, T., and Tibshirani, R. (1989). "Linear Smoothers and Additive Models", ''The Annals of Statistics'' 17(2):453–555. ]
See also
*
Generalized additive model
*
Backfitting algorithm
*
Projection pursuit regression
*
Generalized additive model for location, scale, and shape (GAMLSS)
*
Median polish
*
Projection pursuit
References
Further reading
*Breiman, L. and
Friedman, J.H. (1985). "Estimating Optimal Transformations for Multiple Regression and Correlation", ''
Journal of the American Statistical Association
The ''Journal of the American Statistical Association'' is a quarterly peer-reviewed scientific journal published by Taylor & Francis on behalf of the American Statistical Association. It covers work primarily focused on the application of statis ...
'' 80:580–598. {{doi, 10.1080/01621459.1985.10478157
Nonparametric regression
Regression models