Projection pursuit (PP) is a type of statistical technique that involves finding the most "interesting" possible
projections in multidimensional data. Often, projections that deviate more from a
normal distribution
In probability theory and statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is
f(x) = \frac ...
are considered to be more interesting. As each projection is found, the data are reduced by removing the component along that projection, and the process is repeated to find new projections; this is the "pursuit" aspect that motivated the technique known as
matching pursuit.
The idea of projection pursuit is to locate the projection or projections from
high-dimensional space to low-dimensional space that reveal the most details about the structure of the data set. Once an interesting set of projections has been found, existing structures (clusters, surfaces, etc.) can be extracted and analyzed separately.
Projection pursuit has been widely used for
blind source separation, so it is very important in
independent component analysis
In signal processing, independent component analysis (ICA) is a computational method for separating a multivariate statistics, multivariate signal into additive subcomponents. This is done by assuming that at most one subcomponent is Gaussian and ...
. Projection pursuit seeks one projection at a time such that the extracted signal is as non-Gaussian as possible.
History
Projection pursuit technique were originally proposed and experimented by Kruskal. Related ideas occur in Switzer (1970) "Numerical classification" pp31–43 in "Computer Applications in the Earth Sciences: Geostatistics, and Switzer and Wright (1971) "Numerical classification of eocene nummulitids," Mathematical Geology pp 297–311. The first successful implementation is due to
Jerome H. Friedman and
John Tukey
John Wilder Tukey (; June 16, 1915 – July 26, 2000) was an American mathematician and statistician, best known for the development of the fast Fourier Transform (FFT) algorithm and box plot. The Tukey range test, the Tukey lambda distributi ...
(1974), who named projection pursuit.
The original purpose of projection pursuit was to machine-pick "interesting" low-dimensional projections of a high-dimensional point cloud by numerically maximizing a certain objective function or projection index.
Several years later, Friedman and Stuetzle extended the idea behind projection pursuit and added
projection pursuit regression (PPR), projection pursuit classification (PPC), and projection pursuit density estimation (PPDE).
Feature
The most exciting feature of projection pursuit is that it is one of the very few multivariate methods able to bypass the "
curse of dimensionality" caused by the fact that high-dimensional space is mostly empty. In addition, projection pursuit is able to ignore irrelevant (i.e. noisy and information-poor) variables. This is a distinct advantage over methods based on interpoint distances like minimal spanning trees, multidimensional scaling and most clustering techniques.
Many of the methods of classical multivariate analysis turn out to be special cases of projection pursuit. Examples are
principal component analysis
Principal component analysis (PCA) is a linear dimensionality reduction technique with applications in exploratory data analysis, visualization and data preprocessing.
The data is linearly transformed onto a new coordinate system such that th ...
and
discriminant analysis, and the quartimax and oblimax methods in
factor analysis
Factor analysis is a statistical method used to describe variability among observed, correlated variables in terms of a potentially lower number of unobserved variables called factors. For example, it is possible that variations in six observe ...
.
One serious drawback of projection pursuit methods is their high demand on computer time.
See also
*
Projection pursuit regression
*
Targeted projection pursuit
References
{{Authority control
Exploratory data analysis
Multivariate statistics