Additive Model

In statistics, an additive model (AM) is a nonparametric regression method. It was suggested by Jerome H. Friedman and Werner Stuetzle (1981) and is an essential part of the ACE algorithm. The ''AM'' uses a one-dimensional smoother to build a restricted class of nonparametric regression models. Because of this, it is less affected by the curse of dimensionality than e.g. a ''p''-dimensional smoother. Furthermore, the ''AM'' is more flexible than a standard linear model, while being more interpretable than a general regression surface at the cost of approximation errors. Problems with ''AM'', like many other machine learning methods, include model selection, overfitting, and multicollinearity.

Description

Given a data set $\_^n$ of ''n'' statistical units, where $\_^n$ represent predictors and $y_i$ is the outcome, the ''additive model'' takes the form
: \mathrm x_, \ldots, x_= \beta_0+\sum_^p f_j(x_)
or
: $Y= \beta_0+\sum_^p f_j(X_)+\varepsilon$
Where \mathrm \epsilon = 0, $\mathrm(\epsilon) = \sigma^2$ and \mathrm f_j(X_) = 0. The functions $f_j(x_)$ are unknown smooth functions fit from the data. Fitting the ''AM'' (i.e. the functions $f_j(x_)$) can be done using the backfitting algorithm proposed by Andreas Buja, Trevor Hastie and Robert Tibshirani (1989).Buja, A., Hastie, T., and Tibshirani, R. (1989). "Linear Smoothers and Additive Models", ''The Annals of Statistics'' 17(2):453–555.

See also

* Generalized additive model
* Backfitting algorithm
*Projection pursuit regression
* Generalized additive model for location, scale, and shape (GAMLSS)
* Median polish
*Projection Pursuit

References

Further reading

*Breiman, L. and Friedman, J.H. (1985). "Estimating Optimal Transformations for Multiple Regression and Correlation", ''Journal of the American Statistical Association'' 80:580–598. {{doi, 10.1080/01621459.1985.10478157

Nonparametric regression
Regression models