HOME

TheInfoList



OR:

Multivariate statistics is a subdivision of
statistics Statistics (from German language, German: ', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a s ...
encompassing the simultaneous observation and analysis of more than one outcome variable, i.e., '' multivariate random variables''. Multivariate statistics concerns understanding the different aims and background of each of the different forms of multivariate analysis, and how they relate to each other. The practical application of multivariate statistics to a particular problem may involve several types of univariate and multivariate analyses in order to understand the relationships between variables and their relevance to the problem being studied. In addition, multivariate statistics is concerned with multivariate
probability distribution In probability theory and statistics, a probability distribution is a Function (mathematics), function that gives the probabilities of occurrence of possible events for an Experiment (probability theory), experiment. It is a mathematical descri ...
s, in terms of both :*how these can be used to represent the distributions of observed data; :*how they can be used as part of statistical inference, particularly where several different quantities are of interest to the same analysis. Certain types of problems involving multivariate data, for example
simple linear regression In statistics, simple linear regression (SLR) is a linear regression model with a single explanatory variable. That is, it concerns two-dimensional sample points with one independent variable and one dependent variable (conventionally, the ''x ...
and multiple regression, are ''not'' usually considered to be special cases of multivariate statistics because the analysis is dealt with by considering the (univariate) conditional distribution of a single outcome variable given the other variables.


Multivariate analysis

Multivariate analysis (MVA) is based on the principles of multivariate statistics. Typically, MVA is used to address situations where multiple measurements are made on each experimental unit and the relations among these measurements and their structures are important. A modern, overlapping categorization of MVA includes: * Normal and general multivariate models and distribution theory * The study and measurement of relationships *
Probability Probability is a branch of mathematics and statistics concerning events and numerical descriptions of how likely they are to occur. The probability of an event is a number between 0 and 1; the larger the probability, the more likely an e ...
computations of multidimensional regions * The exploration of
data structures In computer science, a data structure is a data organization and storage format that is usually chosen for efficient access to data. More precisely, a data structure is a collection of data values, the relationships among them, and the functi ...
and patterns Multivariate analysis can be complicated by the desire to include physics-based analysis to calculate the effects of variables for a hierarchical "system-of-systems". Often, studies that wish to use multivariate analysis are stalled by the dimensionality of the problem. These concerns are often eased through the use of surrogate models, highly accurate approximations of the physics-based code. Since surrogate models take the form of an equation, they can be evaluated very quickly. This becomes an enabler for large-scale MVA studies: while a Monte Carlo simulation across the design space is difficult with physics-based codes, it becomes trivial when evaluating surrogate models, which often take the form of response-surface equations.


Types of analysis

Many different models are used in MVA, each with its own type of analysis: # Multivariate analysis of variance (MANOVA) extends the
analysis of variance Analysis of variance (ANOVA) is a family of statistical methods used to compare the Mean, means of two or more groups by analyzing variance. Specifically, ANOVA compares the amount of variation ''between'' the group means to the amount of variati ...
to cover cases where there is more than one dependent variable to be analyzed simultaneously; see also Multivariate analysis of covariance (MANCOVA). #Multivariate regression attempts to determine a formula that can describe how elements in a vector of variables respond simultaneously to changes in others. For linear relations, regression analyses here are based on forms of the
general linear model The general linear model or general multivariate regression model is a compact way of simultaneously writing several multiple linear regression models. In that sense it is not a separate statistical linear model. The various multiple linear regre ...
. Some suggest that multivariate regression is distinct from multivariable regression, however, that is debated and not consistently true across scientific fields. #
Principal components analysis Principal component analysis (PCA) is a Linear map, linear dimensionality reduction technique with applications in exploratory data analysis, visualization and Data Preprocessing, data preprocessing. The data is linear map, linearly transformed ...
(PCA) creates a new set of
orthogonal In mathematics, orthogonality (mathematics), orthogonality is the generalization of the geometric notion of ''perpendicularity''. Although many authors use the two terms ''perpendicular'' and ''orthogonal'' interchangeably, the term ''perpendic ...
variables that contain the same information as the original set. It rotates the axes of variation to give a new set of orthogonal axes, ordered so that they summarize decreasing proportions of the variation. # Factor analysis is similar to PCA but allows the user to extract a specified number of synthetic variables, fewer than the original set, leaving the remaining unexplained variation as error. The extracted variables are known as latent variables or factors; each one may be supposed to account for covariation in a group of observed variables. # Canonical correlation analysis finds linear relationships among two sets of variables; it is the generalised (i.e. canonical) version of bivariate correlation. # Redundancy analysis (RDA) is similar to canonical correlation analysis but allows the user to derive a specified number of synthetic variables from one set of (independent) variables that explain as much variance as possible in another (independent) set. It is a multivariate analogue of regression. # Correspondence analysis (CA), or reciprocal averaging, finds (like PCA) a set of synthetic variables that summarise the original set. The underlying model assumes chi-squared dissimilarities among records (cases). # Canonical (or "constrained") correspondence analysis (CCA) for summarising the joint variation in two sets of variables (like redundancy analysis); combination of correspondence analysis and multivariate regression analysis. The underlying model assumes chi-squared dissimilarities among records (cases). # Multidimensional scaling comprises various algorithms to determine a set of synthetic variables that best represent the pairwise distances between records. The original method is principal coordinates analysis (PCoA; based on PCA). # Discriminant analysis, or canonical variate analysis, attempts to establish whether a set of variables can be used to distinguish between two or more groups of cases. #
Linear discriminant analysis Linear discriminant analysis (LDA), normal discriminant analysis (NDA), canonical variates analysis (CVA), or discriminant function analysis is a generalization of Fisher's linear discriminant, a method used in statistics and other fields, to fi ...
(LDA) computes a linear predictor from two sets of normally distributed data to allow for classification of new observations. # Clustering systems assign objects into groups (called clusters) so that objects (cases) from the same cluster are more similar to each other than objects from different clusters. # Recursive partitioning creates a decision tree that attempts to correctly classify members of the population based on a dichotomous dependent variable. # Artificial neural networks extend regression and clustering methods to non-linear multivariate models. # Statistical graphics such as tours, parallel coordinate plots, scatterplot matrices can be used to explore multivariate data. # Simultaneous equations models involve more than one regression equation, with different dependent variables, estimated together. # Vector autoregression involves simultaneous regressions of various
time series In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. ...
variables on their own and each other's lagged values. # Principal response curves analysis (PRC) is a method based on RDA that allows the user to focus on treatment effects over time by correcting for changes in control treatments over time. # Iconography of correlations consists in replacing a correlation matrix by a diagram where the “remarkable” correlations are represented by a solid line (positive correlation), or a dotted line (negative correlation).


Dealing with incomplete data

It is very common that in an experimentally acquired set of data the values of some components of a given data point are missing. Rather than discarding the whole data point, it is common to "fill in" values for the missing components, a process called " imputation".


Important probability distributions

There is a set of
probability distribution In probability theory and statistics, a probability distribution is a Function (mathematics), function that gives the probabilities of occurrence of possible events for an Experiment (probability theory), experiment. It is a mathematical descri ...
s used in multivariate analyses that play a similar role to the corresponding set of distributions that are used in univariate analysis when the
normal distribution In probability theory and statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is f(x) = \frac ...
is appropriate to a dataset. These multivariate distributions are: :* Multivariate normal distribution :* Wishart distribution :* Multivariate Student-t distribution. The Inverse-Wishart distribution is important in Bayesian inference, for example in Bayesian multivariate linear regression. Additionally, Hotelling's T-squared distribution is a multivariate distribution, generalising Student's t-distribution, that is used in multivariate hypothesis testing.


History

C.R. Rao made significant contributions to multivariate statistical theory throughout his career, particularly in the mid-20th century. One of his key works is the book titled "Advanced Statistical Methods in Biometric Research," published in 1952. This work laid the foundation for many concepts in multivariate statistics. Anderson's 1958 textbook,'' An Introduction to Multivariate Statistical Analysis'', educated a generation of theorists and applied statisticians; Anderson's book emphasizes hypothesis testing via likelihood ratio tests and the properties of power functions: admissibility, unbiasedness and monotonicity. MVA was formerly discussed solely in the context of statistical theories, due to the size and complexity of underlying datasets and its high computational consumption. With the dramatic growth of computational power, MVA now plays an increasingly important role in data analysis and has wide application in
Omics Omics is the collective characterization and quantification of entire sets of biological molecules and the investigation of how they translate into the structure, function, and dynamics of an organism or group of organisms. The branches of scien ...
fields.


Applications

* Multivariate hypothesis testing * Dimensionality reduction * Latent structure discovery * Clustering * Multivariate regression analysis * Classification and discrimination analysis * Variable selection * Multidimensional analysis * Multidimensional scaling * Data mining


Software and tools

There are an enormous number of software packages and other tools for multivariate analysis, including: * JMP (statistical software) * MiniTab * Calc * PSPP * RCRAN
has details on the packages available for multivariate data analysis
* SAS (software) * SciPy for Python * SPSS * Stata * STATISTICA * The Unscrambler * WarpPLS * SmartPLS * MATLAB * Eviews * NCSS (statistical software) includes multivariate analysis.
The Unscrambler® X
is a multivariate analysis tool.
SIMCA
*DataPandit (Free SaaS applications b
Let's Excel Analytics Solutions


See also

* Estimation of covariance matrices * Important publications in multivariate analysis * Multivariate testing in marketing * Structured data analysis (statistics) * Structural equation modeling * RV coefficient * Bivariate analysis * Design of experiments (DoE) * Dimensional analysis *
Exploratory data analysis In statistics, exploratory data analysis (EDA) is an approach of data analysis, analyzing data sets to summarize their main characteristics, often using statistical graphics and other data visualization methods. A statistical model can be used or ...
* OLS * Partial least squares regression *
Pattern recognition Pattern recognition is the task of assigning a class to an observation based on patterns extracted from data. While similar, pattern recognition (PR) is not to be confused with pattern machines (PM) which may possess PR capabilities but their p ...
* Principal component analysis (PCA) * Regression analysis * Soft independent modelling of class analogies (SIMCA) * Statistical interference * Univariate analysis


References


Further reading

* * * A. Sen, M. Srivastava, ''Regression Analysis — Theory, Methods, and Applications'', Springer-Verlag, Berlin, 2011 (4th printing). * * Malakooti, B. (2013). Operations and Production Systems with Multiple Objectives. John Wiley & Sons. * T. W. Anderson, ''An Introduction to Multivariate Statistical Analysis'', Wiley, New York, 1958. * (M.A. level "likelihood" approach) * Feinstein, A. R. (1996) ''Multivariable Analysis''. New Haven, CT: Yale University Press. * Hair, J. F. Jr. (1995) ''Multivariate Data Analysis with Readings'', 4th ed. Prentice-Hall. * Schafer, J. L. (1997) ''Analysis of Incomplete Multivariate Data''. CRC Press. (Advanced) * Sharma, S. (1996) ''Applied Multivariate Techniques''. Wiley. (Informal, applied) * Izenman, Alan J. (2008). Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning. Springer Texts in Statistics. New York: Springer-Verlag. . *


External links


Statnotes: Topics in Multivariate Analysis, by G. David Garson

Mike Palmer: The Ordination Web Page

InsightsNow: Makers of ReportsNow, ProfilesNow, and KnowledgeNow
{{Authority control