HOME

TheInfoList



OR:

In
statistics Statistics (from German: '' Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, indust ...
, probability density estimation or simply density estimation is the construction of an
estimate Estimation (or estimating) is the process of finding an estimate or approximation, which is a value that is usable for some purpose even if input data may be incomplete, uncertain, or unstable. The value is nonetheless usable because it is de ...
, based on observed
data In the pursuit of knowledge, data (; ) is a collection of discrete values that convey information, describing quantity, quality, fact, statistics, other basic units of meaning, or simply sequences of symbols that may be further interpret ...
, of an unobservable underlying
probability density function In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) ca ...
. The unobservable density function is thought of as the density according to which a large population is distributed; the data are usually thought of as a random sample from that population. A variety of approaches to density estimation are used, including Parzen windows and a range of
data clustering Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense) to each other than to those in other groups (clusters). It is a main task of ...
techniques, including
vector quantization Vector quantization (VQ) is a classical quantization technique from signal processing that allows the modeling of probability density functions by the distribution of prototype vectors. It was originally used for data compression. It works by di ...
. The most basic form of density estimation is a rescaled
histogram A histogram is an approximate representation of the frequency distribution, distribution of numerical data. The term was first introduced by Karl Pearson. To construct a histogram, the first step is to "Data binning, bin" (or "Data binning, buck ...
.


Example

We will consider records of the incidence of
diabetes Diabetes, also known as diabetes mellitus, is a group of metabolic disorders characterized by a high blood sugar level ( hyperglycemia) over a prolonged period of time. Symptoms often include frequent urination, increased thirst and increased ...
. The following is quoted verbatim from the
data set A data set (or dataset) is a collection of data. In the case of tabular data, a data set corresponds to one or more database tables, where every column of a table represents a particular variable, and each row corresponds to a given record of the ...
description: :''A population of women who were at least 21 years old, of Pima Indian heritage and living near Phoenix, Arizona, was tested for
diabetes mellitus Diabetes, also known as diabetes mellitus, is a group of metabolic disorders characterized by a high blood sugar level ( hyperglycemia) over a prolonged period of time. Symptoms often include frequent urination, increased thirst and increased ...
according to
World Health Organization The World Health Organization (WHO) is a specialized agency of the United Nations responsible for international public health. The WHO Constitution states its main objective as "the attainment by all peoples of the highest possible level o ...
criteria. The data were collected by the US National Institute of Diabetes and Digestive and Kidney Diseases. We used the 532 complete records.'' In this example, we construct three density estimates for "glu" ( plasma
glucose Glucose is a simple sugar with the molecular formula . Glucose is overall the most abundant monosaccharide, a subcategory of carbohydrates. Glucose is mainly made by plants and most algae during photosynthesis from water and carbon dioxide, u ...
concentration), one
conditional Conditional (if then) may refer to: *Causal conditional, if X then Y, where X is a cause of Y *Conditional probability, the probability of an event A given that another event B has occurred *Conditional proof, in logic: a proof that asserts a co ...
on the presence of diabetes, the second conditional on the absence of diabetes, and the third not conditional on diabetes. The conditional density estimates are then used to construct the probability of diabetes conditional on "glu". The "glu" data were obtained from the MASS package of the
R programming language R is a programming language for statistical computing and graphics supported by the R Core Team and the R Foundation for Statistical Computing. Created by statisticians Ross Ihaka and Robert Gentleman, R is used among data miners, bioinforma ...
. Within R, ?Pima.tr and ?Pima.te give a fuller account of the data. The
mean There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value ( magnitude and sign) of a given data set. For a data set, the '' ar ...
of "glu" in the diabetes cases is 143.1 and the standard deviation is 31.26. The mean of "glu" in the non-diabetes cases is 110.0 and the standard deviation is 24.29. From this we see that, in this data set, diabetes cases are associated with greater levels of "glu". This will be made clearer by plots of the estimated density functions. The first figure shows density estimates of ''p''(glu , diabetes=1), ''p''(glu , diabetes=0), and ''p''(glu). The density estimates are kernel density estimates using a Gaussian kernel. That is, a Gaussian density function is placed at each data point, and the sum of the density functions is computed over the range of the data. From the density of "glu" conditional on diabetes, we can obtain the probability of diabetes conditional on "glu" via
Bayes' rule In probability theory and statistics, Bayes' theorem (alternatively Bayes' law or Bayes' rule), named after Thomas Bayes, describes the probability of an event, based on prior knowledge of conditions that might be related to the event. For exa ...
. For brevity, "diabetes" is abbreviated "db." in this formula. : p(\mbox=1, \mbox) = \frac The second figure shows the estimated posterior probability ''p''(diabetes=1 , glu). From these data, it appears that an increased level of "glu" is associated with diabetes.


Application and Purpose

A very natural use of density estimates is in the informal investigation of the properties of a given set of data. Density estimates can give valuable indication of such features as skewness and multimodality in the data. In some cases they will yield conclusions that may then be regarded as self-evidently true, while in others all they will do is to point the way to further analysis and/or data collection. An important aspect of statistics is often the presentation of data back to the client in order to provide explanation and illustration of conclusions that may possibly have been obtained by other means. Density estimates are ideal for this purpose, for the simple reason that they are fairly easily comprehensible to non-mathematicians. More examples illustrating the use of density estimates for exploratory and presentational purposes, including the important case of bivariate data. Density estimation is also frequently used in anomaly detection or
novelty detection Novelty detection is the mechanism by which an intelligent organism is able to identify an incoming sensory pattern as being hitherto unknown. If the pattern is sufficiently salient or associated with a high positive or strong negative utility, i ...
: if an observation lies in a very low-density region, it is likely to be an anomaly or a novelty. * In
hydrology Hydrology () is the scientific study of the movement, distribution, and management of water on Earth and other planets, including the water cycle, water resources, and environmental watershed sustainability. A practitioner of hydrology is call ...
the
histogram A histogram is an approximate representation of the frequency distribution, distribution of numerical data. The term was first introduced by Karl Pearson. To construct a histogram, the first step is to "Data binning, bin" (or "Data binning, buck ...
and estimated density function of rainfall and river discharge data, analysed with a
probability distribution In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon ...
, are used to gain insight in their behaviour and frequency of occurrence.An illustration of histograms and probability density functions
/ref> An example is shown in the blue figure.


Kernel density estimation


See also

*
Kernel density estimation In statistics, kernel density estimation (KDE) is the application of kernel smoothing for probability density estimation, i.e., a non-parametric method to estimate the probability density function of a random variable based on '' kernels'' as ...
*
Mean integrated squared error In statistics, the mean integrated squared error (MISE) is used in density estimation. The MISE of an estimate of an unknown probability density is given by :\operatorname\, f_n-f\, _2^2=\operatorname\int (f_n(x)-f(x))^2 \, dx where ''ƒ'' is ...
*
Histogram A histogram is an approximate representation of the frequency distribution, distribution of numerical data. The term was first introduced by Karl Pearson. To construct a histogram, the first step is to "Data binning, bin" (or "Data binning, buck ...
*
Multivariate kernel density estimation Kernel density estimation is a nonparametric technique for density estimation i.e., estimation of probability density functions, which is one of the fundamental questions in statistics. It can be viewed as a generalisation of histogram density esti ...
*
Spectral density estimation In statistical signal processing, the goal of spectral density estimation (SDE) or simply spectral estimation is to estimate the spectral density (also known as the power spectral density) of a signal from a sequence of time samples of the sig ...
*
Kernel embedding of distributions In machine learning, the kernel embedding of distributions (also called the kernel mean or mean map) comprises a class of nonparametric methods in which a probability distribution is represented as an element of a reproducing kernel Hilbert spac ...
* Generative model * Application of Order Statistics: Non-parametric Density Estimation *
Probability distribution fitting Probability distribution fitting or simply distribution fitting is the fitting of a probability distribution to a series of data concerning the repeated measurement of a variable phenomenon. The aim of distribution fitting is to predict the proba ...


References

Sources * * Trevor Hastie,
Robert Tibshirani Robert Tibshirani (born July 10, 1956) is a professor in the Departments of Statistics and Biomedical Data Science at Stanford University. He was a professor at the University of Toronto from 1985 to 1998. In his work, he develops statistical ...
, and Jerome Friedman. ''The Elements of Statistical Learning''. New York: Springer, 2001. . ''(See Chapter 6.)'' * Qi Li and Jeffrey S. Racine. ''Nonparametric Econometrics: Theory and Practice''. Princeton University Press, 2007, . ''(See Chapter 1.)'' * D.W. Scott. ''Multivariate Density Estimation. Theory, Practice and Visualization''. New York: Wiley, 1992. * B.W. Silverman. ''Density Estimation''. London: Chapman and Hall, 1986.


External links


CREEM: Centre for Research Into Ecological and Environmental Modelling
Downloads for free density estimation software package
''Distance 4''
(from Research Unit for Wildlife Population Assessment "RUWPA") an
''WiSP''


''(See "Pima Indians Diabetes Database" for the original data set of 732 records, and additional notes.)'' * MATLAB code fo
one dimensional
and
two dimensional
density estimation
libAGF
C++ software for
variable kernel density estimation In statistics, adaptive or "variable-bandwidth" kernel density estimation is a form of kernel density estimation in which the size of the kernels used in the estimate are varied depending upon either the location of the samples or the location of th ...
. {{Statistics, inference * Nonparametric statistics