maximally informative dimensions
   HOME

TheInfoList



OR:

Maximally informative dimensions is a dimensionality reduction technique used in the statistical analyses of neural responses. Specifically, it is a way of projecting a stimulus onto a low-dimensional subspace so that as much
information Information is an abstract concept that refers to that which has the power to inform. At the most fundamental level information pertains to the interpretation of that which may be sensed. Any natural process that is not completely random ...
as possible about the stimulus is preserved in the neural response. It is motivated by the fact that natural stimuli are typically confined by their
statistics Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
to a lower-dimensional space than that spanned by
white noise In signal processing, white noise is a random signal having equal intensity at different frequencies, giving it a constant power spectral density. The term is used, with this or similar meanings, in many scientific and technical disciplines, ...
but correctly identifying this subspace using traditional techniques is complicated by the correlations that exist within natural images. Within this subspace, stimulus-response functions may be either
linear Linearity is the property of a mathematical relationship (''function'') that can be graphically represented as a straight line. Linearity is closely related to '' proportionality''. Examples in physics include rectilinear motion, the linear r ...
or
nonlinear In mathematics and science, a nonlinear system is a system in which the change of the output is not proportional to the change of the input. Nonlinear problems are of interest to engineers, biologists, physicists, mathematicians, and many other ...
. The idea was originally developed by
Tatyana Sharpee Tatyana Sharpee is an American neuroscientist. She is an Associate Professor at the Salk Institute for Biological Studies, where she spearheads a research group at the Computational Neurobiology Laboratory, and serves as the Helen McLorraine Deve ...
, Nicole C. Rust, and
William Bialek William Samuel Bialek (born 1960, in Los Angeles, California) is a theoretical biophysicist and a professor at Princeton University and The Graduate Center, CUNY. Much of his work, which has ranged over a wide variety of theoretical problems at t ...
in 2003.


Mathematical formulation

Neural stimulus-response functions are typically given as the probability of a
neuron A neuron, neurone, or nerve cell is an electrically excitable cell that communicates with other cells via specialized connections called synapses. The neuron is the main component of nervous tissue in all animals except sponges and placozoa. N ...
generating an
action potential An action potential occurs when the membrane potential of a specific cell location rapidly rises and falls. This depolarization then causes adjacent locations to similarly depolarize. Action potentials occur in several types of animal cells, ...
, or spike, in response to a stimulus \mathbf. The goal of maximally informative dimensions is to find a small relevant subspace of the much larger stimulus space that accurately captures the salient features of \mathbf. Let D denote the dimensionality of the entire stimulus space and K denote the dimensionality of the relevant subspace, such that K \ll D. We let \ denote the basis of the relevant subspace, and \mathbf^K the
projection Projection, projections or projective may refer to: Physics * Projection (physics), the action/process of light, heat, or sound reflecting from a surface to another in a different direction * The display of images by a projector Optics, graphic ...
of \mathbf onto \. Using
Bayes' theorem In probability theory and statistics, Bayes' theorem (alternatively Bayes' law or Bayes' rule), named after Thomas Bayes, describes the probability of an event, based on prior knowledge of conditions that might be related to the event. For examp ...
we can write out the probability of a spike given a stimulus: : P(spike, \mathbf^K) = P(spike)f(\mathbf^K) where : f(\mathbf^K) = \frac is some nonlinear function of the projected stimulus. In order to choose the optimal \, we compare the prior stimulus distribution P(\mathbf) with the spike-triggered stimulus distribution P(\mathbf, spike) using the Shannon information. The
average In ordinary language, an average is a single number taken as representative of a list of numbers, usually the sum of the numbers divided by how many numbers are in the list (the arithmetic mean). For example, the average of the numbers 2, 3, 4, 7, ...
information (averaged across all presented stimuli) per spike is given by : I_ = \sum_ P(\mathbf, spike) log_2 spike)/P(\mathbf)/math>.N. Brenner, S. P. Strong, R. Koberle, W. Bialek, and R. R. de Ruyter van Steveninck. "Synergy in a neural code. Neural Comp., 12:1531-1552, 2000. Now consider a K = 1 dimensional subspace defined by a single direction \mathbf. The average information conveyed by a single spike about the projection x = \mathbf \cdot \mathbf is : I(\mathbf) = \int dx P_(x, spike)log2 spike)/P_(x)/math>, where the probability distributions are approximated by a measured data set via P_(x, spike) = \langle \delta(x - \mathbf \cdot \mathbf) , spike \rangle_ and P_(x) = \langle \delta(x - \mathbf \cdot \mathbf)\rangle_, i.e., each presented stimulus is represented by a scaled
Dirac delta function In mathematics, the Dirac delta distribution ( distribution), also known as the unit impulse, is a generalized function or distribution over the real numbers, whose value is zero everywhere except at zero, and whose integral over the entire ...
and the probability distributions are created by averaging over all spike-eliciting stimuli, in the former case, or the entire presented stimulus set, in the latter case. For a given dataset, the average information is a function only of the direction \mathbf. Under this formulation, the relevant subspace of dimension K = 1 would be defined by the direction \mathbf that maximizes the average information I(\mathbf). This procedure can readily be extended to a relevant subspace of dimension K > 1 by defining : P_(\mathbf, spike) = \langle \prod_^K \delta(x_i - \mathbf \cdot \mathbf_i) , spike \rangle_ and : P_(\mathbf) = \langle \prod_^K \delta(x_i - \mathbf \cdot \mathbf_i) \rangle_ and maximizing I().


Importance

Maximally informative dimensions does not make any assumptions about the Gaussianity of the stimulus set, which is important, because naturalistic stimuli tend to have non-Gaussian statistics. In this way the technique is more robust than other dimensionality reduction techniques such as
spike-triggered covariance Spike-triggered covariance (STC) analysis is a tool for characterizing a neuron's response properties using the covariance of stimuli that elicit spikes from a neuron. STC is related to the spike-triggered average (STA), and provides a complementar ...
analyses.


References

{{Reflist Neuroscience Computational neuroscience