artificial neural network In machine learning, a neural network (also artificial neural network or neural net, abbreviated ANN or NN) is a computational model inspired by the structure and functions of biological neural networks. A neural network consists of connected ...

s, a hidden layer is a layer of

artificial neuron An artificial neuron is a mathematical function conceived as a model of a biological neuron in a neural network. The artificial neuron is the elementary unit of an ''artificial neural network''. The design of the artificial neuron was inspired ...

s that is neither an input layer nor an output layer. The simplest examples appear in multilayer perceptrons (MLP), as illustrated in the diagram.{{Cite book , last=Zhang , first=Aston , url= , title=Dive into deep learning , last2=Lipton , first2=Zachary , last3=Li , first3=Mu , last4=Smola , first4=Alexander J. , date=2024 , publisher=Cambridge University Press , isbn=978-1-009-38943-3 , location=Cambridge New York Port Melbourne New Delhi Singapore , chapter=5.1. Multilayer Perceptrons , chapter-url=https://d2l.ai/chapter_multilayer-perceptrons/mlp.html An MLP without any hidden layer is essentially just a

linear model In statistics, the term linear model refers to any model which assumes linearity in the system. The most common occurrence is in connection with regression models and the term is often taken as synonymous with linear regression model. However, t ...

. With hidden layers and

activation function The activation function of a node in an artificial neural network is a function that calculates the output of the node based on its individual inputs and their weights. Nontrivial problems can be solved using only a few nodes if the activation f ...

s, however,

nonlinearity In mathematics and science, a nonlinear system (or a non-linear system) is a system in which the change of the output is not proportional to the change of the input. Nonlinear problems are of interest to engineers, biologists, physicists, mathe ...

is introduced into the model. In typical

machine learning Machine learning (ML) is a field of study in artificial intelligence concerned with the development and study of Computational statistics, statistical algorithms that can learn from data and generalise to unseen data, and thus perform Task ( ...

practice, the weights and biases are initialized, then iteratively updated during training via

backpropagation In machine learning, backpropagation is a gradient computation method commonly used for training a neural network to compute its parameter updates. It is an efficient application of the chain rule to neural networks. Backpropagation computes th ...

References

Deep learning Machine learning