The Helmholtz machine (named after

Hermann von Helmholtz Hermann Ludwig Ferdinand von Helmholtz (; ; 31 August 1821 – 8 September 1894; "von" since 1883) was a German physicist and physician who made significant contributions in several scientific fields, particularly hydrodynamic stability. The ...

and his concept of

Helmholtz free energy In thermodynamics, the Helmholtz free energy (or Helmholtz energy) is a thermodynamic potential that measures the useful work obtainable from a closed thermodynamic system at a constant temperature ( isothermal). The change in the Helmholtz ene ...

) is a type of

artificial neural network In machine learning, a neural network (also artificial neural network or neural net, abbreviated ANN or NN) is a computational model inspired by the structure and functions of biological neural networks. A neural network consists of connected ...

that can account for the hidden structure of a set of data by being trained to create a

generative model In statistical classification, two main approaches are called the generative approach and the discriminative approach. These compute classifiers by different approaches, differing in the degree of statistical modelling. Terminology is inconsiste ...

of the original set of data. The hope is that by learning economical

representations ''Representations'' is an interdisciplinary journal in the humanities published quarterly by the University of California Press. The journal was established in 1983 and is the founding publication of the New Historicism movement of the 1980s. It ...

of the data, the underlying structure of the generative model should reasonably approximate the hidden structure of the data set. A Helmholtz machine contains two networks, a bottom-up ''recognition'' network that takes the data as input and produces a distribution over hidden variables, and a top-down "generative" network that generates values of the hidden variables and the data itself. At the time, Helmholtz machines were one of a handful of learning architectures that used feedback as well as feedforward to ensure quality of learned models. Helmholtz machines are usually trained using an

unsupervised learning Unsupervised learning is a framework in machine learning where, in contrast to supervised learning, algorithms learn patterns exclusively from unlabeled data. Other frameworks in the spectrum of supervisions include weak- or semi-supervision, wh ...

algorithm, such as the

wake-sleep algorithm The wake-sleep algorithm is an unsupervised learning algorithm for deep Generative model, generative models, especially Helmholtz machine, Helmholtz Machines. The algorithm is similar to the Expectation–maximization algorithm, expectation-maximi ...

. They are a precursor to

variational autoencoder In machine learning, a variational autoencoder (VAE) is an artificial neural network architecture introduced by Diederik P. Kingma and Max Welling. It is part of the families of probabilistic graphical models and variational Bayesian metho ...

s, which are instead trained using

backpropagation In machine learning, backpropagation is a gradient computation method commonly used for training a neural network to compute its parameter updates. It is an efficient application of the chain rule to neural networks. Backpropagation computes th ...

. Helmholtz machines may also be used in applications requiring a supervised learning algorithm (e.g. character recognition, or position-invariant recognition of an object within a field).

References

External links

* http://www.cs.utoronto.ca/~hinton/helmholtz.html — Hinton's papers on Helmholtz machines * https://www.nku.edu/~kirby/docs/HelmholtzTutorialKoeln.pdf - A tutorial on Helmholtz machines Artificial neural networks Hermann von Helmholtz {{compu-AI-stub

See also

References

External links