Correction for attenuation
   HOME

TheInfoList



OR:

Regression dilution, also known as regression attenuation, is the
biasing In electronics, biasing is the setting of DC (direct current) operating conditions (current and voltage) of an electronic component that processes time-varying signals. Many electronic devices, such as diodes, transistors and vacuum tubes, wh ...
of the
linear regression In statistics, linear regression is a statistical model, model that estimates the relationship between a Scalar (mathematics), scalar response (dependent variable) and one or more explanatory variables (regressor or independent variable). A mode ...
slope In mathematics, the slope or gradient of a Line (mathematics), line is a number that describes the direction (geometry), direction of the line on a plane (geometry), plane. Often denoted by the letter ''m'', slope is calculated as the ratio of t ...
towards zero (the underestimation of its absolute value), caused by errors in the
independent variable A variable is considered dependent if it depends on (or is hypothesized to depend on) an independent variable. Dependent variables are studied under the supposition or demand that they depend, by some law or rule (e.g., by a mathematical function ...
. Consider fitting a straight line for the relationship of an outcome variable ''y'' to a predictor variable ''x'', and estimating the slope of the line. Statistical variability, measurement error or random noise in the ''y'' variable causes
uncertainty Uncertainty or incertitude refers to situations involving imperfect or unknown information. It applies to predictions of future events, to physical measurements that are already made, or to the unknown, and is particularly relevant for decision ...
in the estimated slope, but not
bias Bias is a disproportionate weight ''in favor of'' or ''against'' an idea or thing, usually in a way that is inaccurate, closed-minded, prejudicial, or unfair. Biases can be innate or learned. People may develop biases for or against an individ ...
: on average, the procedure calculates the right slope. However, variability, measurement error or random noise in the ''x'' variable causes bias in the estimated slope (as well as imprecision). The greater the variance in the ''x'' measurement, the closer the estimated slope must approach zero instead of the true value. It may seem counter-intuitive that noise in the predictor variable ''x'' induces a bias, but noise in the outcome variable ''y'' does not. Recall that linear regression is not symmetric: the line of best fit for predicting ''y'' from ''x'' (the usual linear regression) is not the same as the line of best fit for predicting ''x'' from ''y''.


Slope correction

Regression slope and other regression coefficients can be disattenuated as follows.


The case of a fixed ''x'' variable

The case that ''x'' is fixed, but measured with noise, is known as the ''functional model'' or ''functional relationship''. It can be corrected using
total least squares In applied statistics, total least squares is a type of errors-in-variables regression, a least squares data modeling technique in which observational errors on both dependent and independent variables are taken into account. It is a generaliz ...
and errors-in-variables models in general.


The case of a randomly distributed ''x'' variable

The case that the ''x'' variable arises randomly is known as the ''structural model'' or ''structural relationship''. For example, in a medical study patients are recruited as a sample from a population, and their characteristics such as
blood pressure Blood pressure (BP) is the pressure of Circulatory system, circulating blood against the walls of blood vessels. Most of this pressure results from the heart pumping blood through the circulatory system. When used without qualification, the term ...
may be viewed as arising from a
random sample In this statistics, quality assurance, and survey methodology, sampling is the selection of a subset or a statistical sample (termed sample for short) of individuals from within a statistical population to estimate characteristics of the whole ...
. Under certain assumptions (typically,
normal distribution In probability theory and statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is f(x) = \frac ...
assumptions) there is a known
ratio In mathematics, a ratio () shows how many times one number contains another. For example, if there are eight oranges and six lemons in a bowl of fruit, then the ratio of oranges to lemons is eight to six (that is, 8:6, which is equivalent to the ...
between the true slope, and the expected estimated slope. Frost and Thompson (2000) review several methods for estimating this ratio and hence correcting the estimated slope.Frost, C. and S. Thompson (2000). "Correcting for regression dilution bias: comparison of methods for a single predictor variable."
Journal of the Royal Statistical Society The ''Journal of the Royal Statistical Society'' is a peer-reviewed scientific journal of statistics. It comprises three series and is published by Oxford University Press for the Royal Statistical Society. History The Statistical Society of ...
Series A 163: 173–190.
The term ''regression dilution ratio'', although not defined in quite the same way by all authors, is used for this general approach, in which the usual linear regression is fitted, and then a correction applied. The reply to Frost & Thompson by Longford (2001) refers the reader to other methods, expanding the regression model to acknowledge the variability in the x variable, so that no bias arises. Fuller (1987) is one of the standard references for assessing and correcting for regression dilution. Hughes (1993) shows that the regression dilution ratio methods apply approximately in survival models. Rosner (1992) shows that the ratio methods apply approximately to
logistic regression In statistics, a logistic model (or logit model) is a statistical model that models the logit, log-odds of an event as a linear function (calculus), linear combination of one or more independent variables. In regression analysis, logistic regres ...
models. Carroll et al. (1995) give more detail on regression dilution in nonlinear models, presenting the regression dilution ratio methods as the simplest case of ''regression calibration'' methods, in which additional covariates may also be incorporated.Carroll, R. J., Ruppert, D., and Stefanski, L. A. (1995). Measurement error in non-linear models. New York, Wiley. In general, methods for the structural model require some estimate of the variability of the x variable. This will require repeated measurements of the x variable in the same individuals, either in a sub-study of the main data set, or in a separate data set. Without this information it will not be possible to make a correction.


Multiple ''x'' variables

The case of multiple predictor variables subject to variability (possibly
correlated In statistics, correlation or dependence is any statistical relationship, whether causal or not, between two random variables or bivariate data. Although in the broadest sense, "correlation" may indicate any type of association, in statistic ...
) has been well-studied for linear regression, and for some non-linear regression models. Other non-linear models, such as proportional hazards models for
survival analysis Survival analysis is a branch of statistics for analyzing the expected duration of time until one event occurs, such as death in biological organisms and failure in mechanical systems. This topic is called reliability theory, reliability analysis ...
, have been considered only with a single predictor subject to variability.


Correlation correction

Charles Spearman Charles Edward Spearman, FRS (10 September 1863 – 17 September 1945) was an English psychologist known for work in statistics, as a pioneer of factor analysis, and for Spearman's rank correlation coefficient. He also did seminal work on mod ...
developed in 1904 a procedure for correcting correlations for regression dilution, i.e., to "rid a
correlation In statistics, correlation or dependence is any statistical relationship, whether causal or not, between two random variables or bivariate data. Although in the broadest sense, "correlation" may indicate any type of association, in statistics ...
coefficient from the weakening effect of
measurement error Observational error (or measurement error) is the difference between a measured value of a quantity and its unknown true value.Dodge, Y. (2003) ''The Oxford Dictionary of Statistical Terms'', OUP. Such errors are inherent in the measurement pr ...
". In
measurement Measurement is the quantification of attributes of an object or event, which can be used to compare with other objects or events. In other words, measurement is a process of determining how large or small a physical quantity is as compared to ...
and
statistics Statistics (from German language, German: ', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a s ...
, the procedure is also called correlation disattenuation or the disattenuation of correlation. The correction assures that the
Pearson correlation coefficient In statistics, the Pearson correlation coefficient (PCC) is a correlation coefficient that measures linear correlation between two sets of data. It is the ratio between the covariance of two variables and the product of their standard deviatio ...
across data units (for example, people) between two sets of variables is estimated in a manner that accounts for error contained within the measurement of those variables.


Formulation

Let \beta and \theta be the true values of two attributes of some person or
statistical unit In statistics, a unit is one member of a set of entities being studied. It is the main source for the mathematical abstraction of a "random variable". Common examples of a unit would be a single person, animal, plant, manufactured item, or countr ...
. These values are variables by virtue of the assumption that they differ for different statistical units in the
population Population is a set of humans or other organisms in a given region or area. Governments conduct a census to quantify the resident population size within a given jurisdiction. The term is also applied to non-human animals, microorganisms, and pl ...
. Let \hat and \hat be estimates of \beta and \theta derived either directly by observation-with-error or from application of a measurement model, such as the
Rasch model The Rasch model, named after Georg Rasch, is a psychometric model for analyzing categorical data, such as answers to questions on a reading assessment or questionnaire responses, as a function of the trade-off between the respondent's abilities, ...
. Also, let :: \hat = \beta + \epsilon_ , \quad\quad \hat = \theta + \epsilon_\theta, where \epsilon_ and \epsilon_\theta are the measurement errors associated with the estimates \hat and \hat. The estimated correlation between two sets of estimates is : \operatorname(\hat,\hat)= \frac ::::: =\frac, which, assuming the errors are uncorrelated with each other and with the true attribute values, gives : \operatorname(\hat,\hat)= \frac ::::: =\frac.\frac ::::: =\rho \sqrt, where R_\beta is the ''separation index'' of the set of estimates of \beta, which is analogous to
Cronbach's alpha Cronbach's alpha (Cronbach's \alpha), also known as tau-equivalent reliability (\rho_T) or coefficient alpha (coefficient \alpha), is a reliability coefficient and a measure of the internal consistency of tests and measures. It was named after ...
; that is, in terms of
classical test theory Classical test theory (CTT) is a body of related psychometric theory that predicts outcomes of psychological Test (assessment), testing such as the difficulty of items or the ability of test-takers. It is a theory of testing based on the idea that ...
, R_\beta is analogous to a reliability coefficient. Specifically, the separation index is given as follows: : R_\beta=\frac=\frac, where the mean squared standard error of person estimate gives an estimate of the variance of the errors, \epsilon_\beta. The standard errors are normally produced as a by-product of the estimation process (see Rasch model estimation). The disattenuated estimate of the correlation between the two sets of parameter estimates is therefore : \rho = \frac. That is, the disattenuated correlation estimate is obtained by dividing the correlation between the estimates by the
geometric mean In mathematics, the geometric mean is a mean or average which indicates a central tendency of a finite collection of positive real numbers by using the product of their values (as opposed to the arithmetic mean which uses their sum). The geometri ...
of the separation indices of the two sets of estimates. Expressed in terms of classical test theory, the correlation is divided by the geometric mean of the reliability coefficients of two tests. Given two
random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a Mathematics, mathematical formalization of a quantity or object which depends on randomness, random events. The term 'random variable' in its mathema ...
s X^\prime and Y^\prime measured as X and Y with measured
correlation In statistics, correlation or dependence is any statistical relationship, whether causal or not, between two random variables or bivariate data. Although in the broadest sense, "correlation" may indicate any type of association, in statistics ...
r_ and a known reliability for each variable, r_ and r_, the estimated correlation between X^\prime and Y^\prime corrected for attenuation is :r_ = \frac. How well the variables are measured affects the correlation of ''X'' and ''Y''. The correction for attenuation tells one what the estimated correlation is expected to be if one could measure ''X′'' and ''Y′'' with perfect reliability. Thus if X and Y are taken to be imperfect measurements of underlying variables X' and Y' with independent errors, then r_ estimates the true correlation between X' and Y'.


Applicability

A correction for regression dilution is necessary in
statistical inference Statistical inference is the process of using data analysis to infer properties of an underlying probability distribution.Upton, G., Cook, I. (2008) ''Oxford Dictionary of Statistics'', OUP. . Inferential statistical analysis infers properties of ...
based on regression coefficients. However, in
predictive modelling Predictive modelling uses statistics to Prediction, predict outcomes. Most often the event one wants to predict is in the future, but predictive modelling can be applied to any type of unknown event, regardless of when it occurred. For example, pre ...
applications, correction is neither necessary nor appropriate. In
change detection In statistical analysis, change detection or change point detection tries to identify times when the probability distribution of a stochastic process or time series changes. In general the problem concerns both detecting whether or not a change ...
, correction is necessary. To understand this, consider the measurement error as follows. Let ''y'' be the outcome variable, ''x'' be the true predictor variable, and ''w'' be an approximate observation of ''x''. Frost and Thompson suggest, for example, that ''x'' may be the true, long-term blood pressure of a patient, and ''w'' may be the blood pressure observed on one particular clinic visit. Regression dilution arises if we are interested in the relationship between ''y'' and ''x'', but estimate the relationship between ''y'' and ''w''. Because ''w'' is measured with variability, the slope of a regression line of ''y'' on ''w'' is less than the regression line of ''y'' on ''x''. Standard methods can fit a regression of y on w without bias. There is bias only if we then use the regression of y on w as an approximation to the regression of y on x. In the example, assuming that blood pressure measurements are similarly variable in future patients, our regression line of y on w (observed blood pressure) gives unbiased predictions. An example of a circumstance in which correction is desired is prediction of change. Suppose the change in ''x'' is known under some new circumstance: to estimate the likely change in an outcome variable ''y'', the slope of the regression of ''y'' on ''x'' is needed, not ''y'' on ''w''. This arises in
epidemiology Epidemiology is the study and analysis of the distribution (who, when, and where), patterns and Risk factor (epidemiology), determinants of health and disease conditions in a defined population, and application of this knowledge to prevent dise ...
. To continue the example in which ''x'' denotes blood pressure, perhaps a large
clinical trial Clinical trials are prospective biomedical or behavioral research studies on human subject research, human participants designed to answer specific questions about biomedical or behavioral interventions, including new treatments (such as novel v ...
has provided an estimate of the change in blood pressure under a new treatment; then the possible effect on ''y'', under the new treatment, should be estimated from the slope in the regression of ''y'' on ''x''. Another circumstance is predictive modelling in which future observations are also variable, but not (in the phrase used above) "similarly variable". For example, if the current data set includes blood pressure measured with greater precision than is common in clinical practice. One specific example of this arose when developing a regression equation based on a clinical trial, in which blood pressure was the average of six measurements, for use in clinical practice, where blood pressure is usually a single measurement. All of these results can be shown mathematically, in the case of
simple linear regression In statistics, simple linear regression (SLR) is a linear regression model with a single explanatory variable. That is, it concerns two-dimensional sample points with one independent variable and one dependent variable (conventionally, the ''x ...
assuming normal distributions throughout (the framework of Frost & Thompson). It has been discussed that a poorly executed correction for regression dilution, in particular when performed without checking for the underlying assumptions, may do more damage to an estimate than no correction.


Further reading

Regression dilution was first mentioned, under the name attenuation, by Spearman (1904). Those seeking a readable mathematical treatment might like to start with Frost and Thompson (2000).


See also

* Errors-in-variables models *
Quantization (signal processing) Quantization, in mathematics and digital signal processing, is the process of mapping input values from a large set (often a continuous set) to output values in a (countable) smaller set, often with a finite number of elements. Rounding and tr ...
– a common source of error in the explanatory or independent variables


References

{{DEFAULTSORT:Regression Dilution Regression models