HOME

TheInfoList



OR:

In
statistics Statistics (from German: '' Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, indust ...
, a generalized linear model (GLM) is a flexible generalization of ordinary linear regression. The GLM generalizes linear regression by allowing the linear model to be related to the response variable via a ''link function'' and by allowing the magnitude of the variance of each measurement to be a function of its predicted value. Generalized linear models were formulated by
John Nelder John Ashworth Nelder (8 October 1924 – 7 August 2010) was a British statistician known for his contributions to experimental design, analysis of variance, computational statistics, and statistical theory. Contributions Nelder's work was infl ...
and Robert Wedderburn as a way of unifying various other statistical models, including linear regression,
logistic regression In statistics, the logistic model (or logit model) is a statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables. In regression an ...
and Poisson regression. They proposed an
iteratively reweighted least squares The method of iteratively reweighted least squares (IRLS) is used to solve certain optimization problems with objective functions of the form of a ''p''-norm: :\underset \sum_^n \big, y_i - f_i (\boldsymbol\beta) \big, ^p, by an iterative met ...
method for maximum likelihood estimation (MLE) of the model parameters. MLE remains popular and is the default method on many statistical computing packages. Other approaches, including Bayesian regression and least squares fitting to variance stabilized responses, have been developed.


Intuition

Ordinary linear regression predicts the expected value of a given unknown quantity (the ''response variable'', a random variable) as a linear combination of a set of observed values (''predictors''). This implies that a constant change in a predictor leads to a constant change in the response variable (i.e. a ''linear-response model''). This is appropriate when the response variable can vary, to a good approximation, indefinitely in either direction, or more generally for any quantity that only varies by a relatively small amount compared to the variation in the predictive variables, e.g. human heights. However, these assumptions are inappropriate for some types of response variables. For example, in cases where the response variable is expected to be always positive and varying over a wide range, constant input changes lead to geometrically (i.e. exponentially) varying, rather than constantly varying, output changes. As an example, suppose a linear prediction model learns from some data (perhaps primarily drawn from large beaches) that a 10 degree temperature decrease would lead to 1,000 fewer people visiting the beach. This model is unlikely to generalize well over different sized beaches. More specifically, the problem is that if you use the model to predict the new attendance with a temperature drop of 10 for a beach that regularly receives 50 beachgoers, you would predict an impossible attendance value of −950. Logically, a more realistic model would instead predict a constant ''rate'' of increased beach attendance (e.g. an increase of 10 degrees leads to a doubling in beach attendance, and a drop of 10 degrees leads to a halving in attendance). Such a model is termed an ''exponential-response model'' (or ''
log-linear model A log-linear model is a mathematical model that takes the form of a function whose logarithm equals a linear combination of the parameters of the model, which makes it possible to apply (possibly multivariate) linear regression. That is, it h ...
'', since the
logarithm In mathematics, the logarithm is the inverse function to exponentiation. That means the logarithm of a number  to the base  is the exponent to which must be raised, to produce . For example, since , the ''logarithm base'' 10 ...
of the response is predicted to vary linearly). Similarly, a model that predicts a probability of making a yes/no choice (a Bernoulli variable) is even less suitable as a linear-response model, since probabilities are bounded on both ends (they must be between 0 and 1). Imagine, for example, a model that predicts the likelihood of a given person going to the beach as a function of temperature. A reasonable model might predict, for example, that a change in 10 degrees makes a person two times more or less likely to go to the beach. But what does "twice as likely" mean in terms of a probability? It cannot literally mean to double the probability value (e.g. 50% becomes 100%, 75% becomes 150%, etc.). Rather, it is the ''
odds Odds provide a measure of the likelihood of a particular outcome. They are calculated as the ratio of the number of events that produce that outcome to the number that do not. Odds are commonly used in gambling and statistics. Odds also have ...
'' that are doubling: from 2:1 odds, to 4:1 odds, to 8:1 odds, etc. Such a model is a ''log-odds or logistic model''. Generalized linear models cover all these situations by allowing for response variables that have arbitrary distributions (rather than simply
normal distribution In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is : f(x) = \frac e^ The parameter \mu ...
s), and for an arbitrary function of the response variable (the ''link function'') to vary linearly with the predictors (rather than assuming that the response itself must vary linearly). For example, the case above of predicted number of beach attendees would typically be modeled with a Poisson distribution and a log link, while the case of predicted probability of beach attendance would typically be modelled with a
Bernoulli distribution In probability theory and statistics, the Bernoulli distribution, named after Swiss mathematician Jacob Bernoulli,James Victor Uspensky: ''Introduction to Mathematical Probability'', McGraw-Hill, New York 1937, page 45 is the discrete probabi ...
(or binomial distribution, depending on exactly how the problem is phrased) and a log-odds (or ''
logit In statistics, the logit ( ) function is the quantile function associated with the standard logistic distribution. It has many uses in data analysis and machine learning, especially in data transformations. Mathematically, the logit is the ...
'') link function.


Overview

In a generalized linear model (GLM), each outcome Y of the dependent variables is assumed to be generated from a particular
distribution Distribution may refer to: Mathematics * Distribution (mathematics), generalized functions used to formulate solutions of partial differential equations *Probability distribution, the probability of a particular value or value range of a vari ...
in an exponential family, a large class of probability distributions that includes the normal,
binomial Binomial may refer to: In mathematics *Binomial (polynomial), a polynomial with two terms *Binomial coefficient, numbers appearing in the expansions of powers of binomials *Binomial QMF, a perfect-reconstruction orthogonal wavelet decomposition * ...
, Poisson and
gamma Gamma (uppercase , lowercase ; ''gámma'') is the third letter of the Greek alphabet. In the system of Greek numerals it has a value of 3. In Ancient Greek, the letter gamma represented a voiced velar stop . In Modern Greek, this letter r ...
distributions, among others. The mean, ''μ'', of the distribution depends on the independent variables, X, through: : \operatorname(\mathbf, \mathbf) = \boldsymbol = g^(\mathbf\boldsymbol) where E(Y, X) is the expected value of Y
conditional Conditional (if then) may refer to: *Causal conditional, if X then Y, where X is a cause of Y *Conditional probability, the probability of an event A given that another event B has occurred *Conditional proof, in logic: a proof that asserts a co ...
on X; X''β'' is the ''linear predictor'', a linear combination of unknown parameters ''β''; ''g'' is the link function. In this framework, the variance is typically a function, V, of the mean: : \operatorname(\mathbf, \mathbf) = \operatorname(g^(\mathbf\boldsymbol)). It is convenient if V follows from an exponential family of distributions, but it may simply be that the variance is a function of the predicted value. The unknown parameters, ''β'', are typically estimated with
maximum likelihood In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed stat ...
, maximum
quasi-likelihood In statistics, quasi-likelihood methods are used to estimate parameters in a statistical model when exact likelihood methods, for example maximum likelihood estimation, are computationally infeasible. Due to the wrong likelihood being used, quasi- ...
, or Bayesian techniques.


Model components

The GLM consists of three elements: : 1. A particular distribution for modeling Y from among those which are considered exponential families of probability distributions, : 2. A linear predictor \eta = X \beta, and : 3. A link function g such that \operatorname(Y \mid X) = \mu = g^(\eta).


Probability distribution

An overdispersed exponential family of distributions is a generalization of an exponential family and the exponential dispersion model of distributions and includes those families of probability distributions, parameterized by \boldsymbol\theta and \tau, whose density functions ''f'' (or probability mass function, for the case of a discrete distribution) can be expressed in the form : f_Y(\mathbf \mid \boldsymbol\theta, \tau) = h(\mathbf,\tau) \exp \left(\frac \right). \,\! The ''dispersion parameter'', \tau, typically is known and is usually related to the variance of the distribution. The functions h(\mathbf,\tau), \mathbf(\boldsymbol\theta), \mathbf(\mathbf), A(\boldsymbol\theta), and d(\tau) are known. Many common distributions are in this family, including the normal, exponential, gamma, Poisson, Bernoulli, and (for fixed number of trials) binomial, multinomial, and negative binomial. For scalar \mathbf and \boldsymbol\theta (denoted y and \theta in this case), this reduces to : f_Y(y \mid \theta, \tau) = h(y,\tau) \exp \left(\frac \right). \,\! \boldsymbol\theta is related to the mean of the distribution. If \mathbf(\boldsymbol\theta) is the identity function, then the distribution is said to be in canonical form (or ''natural form''). Note that any distribution can be converted to canonical form by rewriting \boldsymbol\theta as \boldsymbol\theta' and then applying the transformation \boldsymbol\theta = \mathbf(\boldsymbol\theta'). It is always possible to convert A(\boldsymbol\theta) in terms of the new parametrization, even if \mathbf(\boldsymbol\theta') is not a one-to-one function; see comments in the page on exponential families. If, in addition, \mathbf(\mathbf) is the identity and \tau is known, then \boldsymbol\theta is called the ''canonical parameter'' (or ''natural parameter'') and is related to the mean through : \boldsymbol\mu = \operatorname(\mathbf) = \nabla A(\boldsymbol\theta). \,\! For scalar \mathbf and \boldsymbol\theta, this reduces to : \mu = \operatorname(y) = A'(\theta). Under this scenario, the variance of the distribution can be shown to be :\operatorname(\mathbf) = \nabla^2 A(\boldsymbol\theta) d(\tau). \,\! For scalar \mathbf and \boldsymbol\theta, this reduces to :\operatorname(y) = A''(\theta) d(\tau). \,\!


Linear predictor

The linear predictor is the quantity which incorporates the information about the independent variables into the model. The symbol ''η'' ( Greek "
eta Eta (uppercase , lowercase ; grc, ἦτα ''ē̂ta'' or ell, ήτα ''ita'' ) is the seventh letter of the Greek alphabet, representing the close front unrounded vowel . Originally denoting the voiceless glottal fricative in most dialects, ...
") denotes a linear predictor. It is related to the expected value of the data through the link function. ''η'' is expressed as linear combinations (thus, "linear") of unknown parameters ''β''. The coefficients of the linear combination are represented as the matrix of independent variables X. ''η'' can thus be expressed as : \eta = \mathbf\boldsymbol.\,


Link function

The link function provides the relationship between the linear predictor and the
mean There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value ( magnitude and sign) of a given data set. For a data set, the '' ar ...
of the distribution function. There are many commonly used link functions, and their choice is informed by several considerations. There is always a well-defined ''canonical'' link function which is derived from the exponential of the response's
density function In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) can ...
. However, in some cases it makes sense to try to match the
domain Domain may refer to: Mathematics *Domain of a function, the set of input values for which the (total) function is defined ** Domain of definition of a partial function ** Natural domain of a partial function **Domain of holomorphy of a function * ...
of the link function to the range of the distribution function's mean, or use a non-canonical link function for algorithmic purposes, for example Bayesian probit regression. When using a distribution function with a canonical parameter \theta, the canonical link function is the function that expresses \theta in terms of \mu, i.e. \theta = b(\mu). For the most common distributions, the mean \mu is one of the parameters in the standard form of the distribution's
density function In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) can ...
, and then b(\mu) is the function as defined above that maps the density function into its canonical form. When using the canonical link function, b(\mu) = \theta = \mathbf\boldsymbol, which allows \mathbf^ \mathbf to be a
sufficient statistic In statistics, a statistic is ''sufficient'' with respect to a statistical model and its associated unknown parameter if "no other statistic that can be calculated from the same sample provides any additional information as to the value of the para ...
for \boldsymbol. Following is a table of several exponential-family distributions in common use and the data they are typically used for, along with the canonical link functions and their inverses (sometimes referred to as the mean function, as done here). In the cases of the exponential and gamma distributions, the domain of the canonical link function is not the same as the permitted range of the mean. In particular, the linear predictor may be positive, which would give an impossible negative mean. When maximizing the likelihood, precautions must be taken to avoid this. An alternative is to use a noncanonical link function. In the case of the Bernoulli, binomial, categorical and multinomial distributions, the support of the distributions is not the same type of data as the parameter being predicted. In all of these cases, the predicted parameter is one or more probabilities, i.e. real numbers in the range ,1/math>. The resulting model is known as ''
logistic regression In statistics, the logistic model (or logit model) is a statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables. In regression an ...
'' (or ''
multinomial logistic regression In statistics, multinomial logistic regression is a classification method that generalizes logistic regression to multiclass problems, i.e. with more than two possible discrete outcomes. That is, it is a model that is used to predict the prob ...
'' in the case that K-way rather than binary values are being predicted). For the Bernoulli and binomial distributions, the parameter is a single probability, indicating the likelihood of occurrence of a single event. The Bernoulli still satisfies the basic condition of the generalized linear model in that, even though a single outcome will always be either 0 or 1, the '' expected value'' will nonetheless be a real-valued probability, i.e. the probability of occurrence of a "yes" (or 1) outcome. Similarly, in a binomial distribution, the expected value is ''Np'', i.e. the expected proportion of "yes" outcomes will be the probability to be predicted. For categorical and multinomial distributions, the parameter to be predicted is a ''K''-vector of probabilities, with the further restriction that all probabilities must add up to 1. Each probability indicates the likelihood of occurrence of one of the ''K'' possible values. For the multinomial distribution, and for the vector form of the categorical distribution, the expected values of the elements of the vector can be related to the predicted probabilities similarly to the binomial and Bernoulli distributions.


Fitting


Maximum likelihood

The
maximum likelihood In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed stat ...
estimates can be found using an
iteratively reweighted least squares The method of iteratively reweighted least squares (IRLS) is used to solve certain optimization problems with objective functions of the form of a ''p''-norm: :\underset \sum_^n \big, y_i - f_i (\boldsymbol\beta) \big, ^p, by an iterative met ...
algorithm or a Newton's method with updates of the form: : \boldsymbol\beta^ = \boldsymbol\beta^ + \mathcal^(\boldsymbol\beta^) u(\boldsymbol\beta^), where \mathcal(\boldsymbol\beta^) is the observed information matrix (the negative of the Hessian matrix) and u(\boldsymbol\beta^) is the score function; or a Fisher's scoring method: : \boldsymbol\beta^ = \boldsymbol\beta^ + \mathcal^(\boldsymbol\beta^) u(\boldsymbol\beta^), where \mathcal(\boldsymbol\beta^) is the Fisher information matrix. Note that if the canonical link function is used, then they are the same.


Bayesian methods

In general, the posterior distribution cannot be found in closed form and so must be approximated, usually using
Laplace approximation In mathematics, Laplace's approximation fits an un-normalised Gaussian approximation to a (twice differentiable) un-normalised target density. In Bayesian statistical inference this is useful to simultaneously approximate the posterior and the ...
s or some type of
Markov chain Monte Carlo In statistics, Markov chain Monte Carlo (MCMC) methods comprise a class of algorithms for sampling from a probability distribution. By constructing a Markov chain that has the desired distribution as its equilibrium distribution, one can obtain ...
method such as
Gibbs sampling In statistics, Gibbs sampling or a Gibbs sampler is a Markov chain Monte Carlo (MCMC) algorithm for obtaining a sequence of observations which are approximated from a specified multivariate probability distribution, when direct sampling is diff ...
.


Examples


General linear models

A possible point of confusion has to do with the distinction between generalized linear models and general linear models, two broad statistical models. Co-originator
John Nelder John Ashworth Nelder (8 October 1924 – 7 August 2010) was a British statistician known for his contributions to experimental design, analysis of variance, computational statistics, and statistical theory. Contributions Nelder's work was infl ...
has expressed regret over this terminology. The general linear model may be viewed as a special case of the generalized linear model with identity link and responses normally distributed. As most exact results of interest are obtained only for the general linear model, the general linear model has undergone a somewhat longer historical development. Results for the generalized linear model with non-identity link are
asymptotic In analytic geometry, an asymptote () of a curve is a line such that the distance between the curve and the line approaches zero as one or both of the ''x'' or ''y'' coordinates tends to infinity. In projective geometry and related context ...
(tending to work well with large samples).


Linear regression

A simple, very important example of a generalized linear model (also an example of a general linear model) is linear regression. In linear regression, the use of the
least-squares The method of least squares is a standard approach in regression analysis to approximate the solution of overdetermined systems (sets of equations in which there are more equations than unknowns) by minimizing the sum of the squares of the res ...
estimator is justified by the Gauss–Markov theorem, which does not assume that the distribution is normal. From the perspective of generalized linear models, however, it is useful to suppose that the distribution function is the normal distribution with constant variance and the link function is the identity, which is the canonical link if the variance is known. Under these assumptions, the least-squares estimator is obtained as the maximum-likelihood parameter estimate. For the normal distribution, the generalized linear model has a closed form expression for the maximum-likelihood estimates, which is convenient. Most other GLMs lack closed form estimates.


Binary data

When the response data, ''Y'', are binary (taking on only values 0 and 1), the distribution function is generally chosen to be the
Bernoulli distribution In probability theory and statistics, the Bernoulli distribution, named after Swiss mathematician Jacob Bernoulli,James Victor Uspensky: ''Introduction to Mathematical Probability'', McGraw-Hill, New York 1937, page 45 is the discrete probabi ...
and the interpretation of ''μ''i is then the probability, ''p'', of ''Y''i taking on the value one. There are several popular link functions for binomial functions.


Logit link function

The most typical link function is the canonical
logit In statistics, the logit ( ) function is the quantile function associated with the standard logistic distribution. It has many uses in data analysis and machine learning, especially in data transformations. Mathematically, the logit is the ...
link: :g(p) = \ln \left( \right). GLMs with this setup are
logistic regression In statistics, the logistic model (or logit model) is a statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables. In regression an ...
models (or ''logit models'').


Probit link function as popular choice of inverse cumulative distribution function

Alternatively, the inverse of any continuous cumulative distribution function (CDF) can be used for the link since the CDF's range is ,1/math>, the range of the binomial mean. The normal CDF \Phi is a popular choice and yields the probit model. Its link is :g(p) = \Phi^(p).\,\! The reason for the use of the probit model is that a constant scaling of the input variable to a normal CDF (which can be absorbed through equivalent scaling of all of the parameters) yields a function that is practically identical to the logit function, but probit models are more tractable in some situations than logit models. (In a Bayesian setting in which normally distributed
prior distribution In Bayesian statistical inference, a prior probability distribution, often simply called the prior, of an uncertain quantity is the probability distribution that would express one's beliefs about this quantity before some evidence is taken into ...
s are placed on the parameters, the relationship between the normal priors and the normal CDF link function means that a probit model can be computed using
Gibbs sampling In statistics, Gibbs sampling or a Gibbs sampler is a Markov chain Monte Carlo (MCMC) algorithm for obtaining a sequence of observations which are approximated from a specified multivariate probability distribution, when direct sampling is diff ...
, while a logit model generally cannot.)


Complementary log-log (cloglog)

The complementary log-log function may also be used: :g(p) = \log(-\log(1-p)). This link function is asymmetric and will often produce different results from the logit and probit link functions. The cloglog model corresponds to applications where we observe either zero events (e.g., defects) or one or more, where the number of events is assumed to follow the Poisson distribution. The Poisson assumption means that :\Pr(0) = \exp(-\mu), where ''μ'' is a positive number denoting the expected number of events. If ''p'' represents the proportion of observations with at least one event, its complement :(1-p) = \Pr(0) = \exp(-\mu), and then :(-\log(1-p)) = \mu. A linear model requires the response variable to take values over the entire real line. Since ''μ'' must be positive, we can enforce that by taking the logarithm, and letting log(''μ'') be a linear model. This produces the "cloglog" transformation :\log(-\log(1-p)) = \log(\mu).


Identity link

The identity link ''g(p) = p'' is also sometimes used for binomial data to yield a
linear probability model In statistics, a linear probability model (LPM) is a special case of a binary regression model. Here the dependent variable for each observation takes values which are either 0 or 1. The probability of observing a 0 or 1 in any one case is treated ...
. However, the identity link can predict nonsense "probabilities" less than zero or greater than one. This can be avoided by using a transformation like cloglog, probit or logit (or any inverse cumulative distribution function). A primary merit of the identity link is that it can be estimated using linear math—and other standard link functions are approximately linear matching the identity link near ''p'' = 0.5.


Variance function

The
variance function In statistics, the variance function is a smooth function which depicts the variance of a random quantity as a function of its mean. The variance function is a measure of heteroscedasticity and plays a large role in many settings of statis ...
for "" data is: :\operatorname(Y_i)= \tau\mu_i (1-\mu_i)\,\! where the dispersion parameter ''τ'' is exactly 1 for the binomial distribution. Indeed, the standard binomial likelihood omits ''τ''. When it is present, the model is called "quasibinomial", and the modified likelihood is called a
quasi-likelihood In statistics, quasi-likelihood methods are used to estimate parameters in a statistical model when exact likelihood methods, for example maximum likelihood estimation, are computationally infeasible. Due to the wrong likelihood being used, quasi- ...
, since it is not generally the likelihood corresponding to any real family of probability distributions. If ''τ'' exceeds 1, the model is said to exhibit
overdispersion In statistics, overdispersion is the presence of greater variability ( statistical dispersion) in a data set than would be expected based on a given statistical model. A common task in applied statistics is choosing a parametric model to fit a ...
.


Multinomial regression

The binomial case may be easily extended to allow for a multinomial distribution as the response (also, a Generalized Linear Model for counts, with a constrained total). There are two ways in which this is usually done:


Ordered response

If the response variable is ordinal, then one may fit a model function of the form: : g(\mu_m) = \eta_m = \beta_0 + X_1 \beta_1 + \cdots + X_p \beta_p + \gamma_2 + \cdots + \gamma_m = \eta_1 + \gamma_2 + \cdots + \gamma_m \text \mu_m = \operatorname(Y \leq m). \, for ''m'' > 2. Different links ''g'' lead to
ordinal regression In statistics, ordinal regression, also called ordinal classification, is a type of regression analysis used for predicting an ordinal variable, i.e. a variable whose value exists on an arbitrary scale where only the relative ordering between diff ...
models like proportional odds models or
ordered probit In statistics, ordered probit is a generalization of the widely used probit analysis to the case of more than two outcomes of an ordinal dependent variable (a dependent variable for which the potential values have a natural ordering, as in poor, ...
models.


Unordered response

If the response variable is a nominal measurement, or the data do not satisfy the assumptions of an ordered model, one may fit a model of the following form: : g(\mu_m) = \eta_m = \beta_ + X_1 \beta_ + \cdots + X_p \beta_ \text \mu_m = \mathrm(Y = m \mid Y \in \ ). \, for ''m'' > 2. Different links ''g'' lead to multinomial logit or
multinomial probit In statistics and econometrics, the multinomial probit model is a generalization of the probit model used when there are several possible categories that the dependent variable can fall into. As such, it is an alternative to the multinomial lo ...
models. These are more general than the ordered response models, and more parameters are estimated.


Count data

Another example of generalized linear models includes Poisson regression which models count data using the Poisson distribution. The link is typically the logarithm, the canonical link. The variance function is proportional to the mean :\operatorname(Y_i) = \tau\mu_i,\, where the dispersion parameter ''τ'' is typically fixed at exactly one. When it is not, the resulting
quasi-likelihood In statistics, quasi-likelihood methods are used to estimate parameters in a statistical model when exact likelihood methods, for example maximum likelihood estimation, are computationally infeasible. Due to the wrong likelihood being used, quasi- ...
model is often described as Poisson with
overdispersion In statistics, overdispersion is the presence of greater variability ( statistical dispersion) in a data set than would be expected based on a given statistical model. A common task in applied statistics is choosing a parametric model to fit a ...
or ''quasi-Poisson''.


Extensions


Correlated or clustered data

The standard GLM assumes that the observations are
uncorrelated In probability theory and statistics, two real-valued random variables, X, Y, are said to be uncorrelated if their covariance, \operatorname ,Y= \operatorname Y- \operatorname \operatorname /math>, is zero. If two variables are uncorrelated, ther ...
. Extensions have been developed to allow for correlation between observations, as occurs for example in longitudinal studies and clustered designs: * Generalized estimating equations (GEEs) allow for the correlation between observations without the use of an explicit probability model for the origin of the correlations, so there is no explicit likelihood. They are suitable when the random effects and their variances are not of inherent interest, as they allow for the correlation without explaining its origin. The focus is on estimating the average response over the population ("population-averaged" effects) rather than the regression parameters that would enable prediction of the effect of changing one or more components of X on a given individual. GEEs are usually used in conjunction with Huber–White standard errors. * Generalized linear mixed models (GLMMs) are an extension to GLMs that includes random effects in the linear predictor, giving an explicit probability model that explains the origin of the correlations. The resulting "subject-specific" parameter estimates are suitable when the focus is on estimating the effect of changing one or more components of X on a given individual. GLMMs are also referred to as
multilevel model Multilevel models (also known as hierarchical linear models, linear mixed-effect model, mixed models, nested data models, random coefficient, random-effects models, random parameter models, or split-plot designs) are statistical models of parame ...
s and as mixed model. In general, fitting GLMMs is more computationally complex and intensive than fitting GEEs.


Generalized additive models

Generalized additive models (GAMs) are another extension to GLMs in which the linear predictor ''η'' is not restricted to be linear in the covariates X but is the sum of smoothing functions applied to the ''xi''s: : \eta = \beta_0 + f_1(x_1) + f_2(x_2) + \cdots \,\! The smoothing functions ''fi'' are estimated from the data. In general this requires a large number of data points and is computationally intensive.


See also

* * * * * * * * * * (VGLM)


References


Citations


Bibliography

* * * *


Further reading

* * *


External links

* {{DEFAULTSORT:Generalized Linear Model Actuarial science Regression models