neuroscience Neuroscience is the scientific study of the nervous system (the brain, spinal cord, and peripheral nervous system), its functions, and its disorders. It is a multidisciplinary science that combines physiology, anatomy, molecular biology, ...

and

computer science Computer science is the study of computation, information, and automation. Computer science spans Theoretical computer science, theoretical disciplines (such as algorithms, theory of computation, and information theory) to Applied science, ...

, synaptic weight refers to the strength or

amplitude The amplitude of a periodic variable is a measure of its change in a single period (such as time or spatial period). The amplitude of a non-periodic signal is its magnitude compared with a reference value. There are various definitions of am ...

of a connection between two nodes, corresponding in biology to the amount of influence the

firing Firing may refer to: * Dismissal (employment), sudden loss of employment by termination * Firemaking, the act of starting a fire * Burning; see combustion * Shooting, specifically the discharge of firearms * Execution by firing squad, a method ...

of one

neuron A neuron (American English), neurone (British English), or nerve cell, is an membrane potential#Cell excitability, excitable cell (biology), cell that fires electric signals called action potentials across a neural network (biology), neural net ...

has on another. The term is typically used in

artificial Artificiality (the state of being artificial, anthropogenic, or man-made) is the state of being the product of intentional human manufacture, rather than occurring naturally through processes not involving or requiring human activity. Connotati ...

and

biological Biology is the scientific study of life and living organisms. It is a broad natural science that encompasses a wide range of fields and unifying principles that explain the structure, function, growth, origin, evolution, and distribution of ...

neural network A neural network is a group of interconnected units called neurons that send signals to one another. Neurons can be either biological cells or signal pathways. While individual neurons are simple, many of them together in a network can perfor ...

research.

Computation

In a computational neural network, a

vector Vector most often refers to: * Euclidean vector, a quantity with a magnitude and a direction * Disease vector, an agent that carries and transmits an infectious pathogen into another living organism Vector may also refer to: Mathematics a ...

or set of inputs

\textbf

and outputs

\textbf

, or pre- and post-synaptic neurons respectively, are interconnected with synaptic weights represented by the matrix

w

, where for a linear neuron :

y_j = \sum_i w_ x_i ~~\textrm~~ \textbf = w\textbf

. where the rows of the synaptic matrix represent the vector of synaptic weights for the output indexed by

j

. The synaptic weight is changed by using a learning rule, the most basic of which is Hebb's rule, which is usually stated in biological terms as

''Neurons that fire together, wire together.''

Computationally, this means that if a large signal from one of the input neurons results in a large signal from one of the output neurons, then the synaptic weight between those two neurons will increase. The rule is unstable, however, and is typically modified using such variations as Oja's rule, radial basis functions or the

backpropagation In machine learning, backpropagation is a gradient computation method commonly used for training a neural network to compute its parameter updates. It is an efficient application of the chain rule to neural networks. Backpropagation computes th ...

algorithm.

Biology

For biological networks, the effect of synaptic weights is not as simple as for linear neurons or

Hebbian learning Hebbian theory is a neuropsychological theory claiming that an increase in synaptic efficacy arises from a presynaptic cell's repeated and persistent stimulation of a postsynaptic cell. It is an attempt to explain synaptic plasticity, the adaptat ...

. However,

biophysical Biophysics is an interdisciplinary science that applies approaches and methods traditionally used in physics to study biological phenomena. Biophysics covers all scales of biological organization, from molecular to organismic and populations. B ...

models such as

BCM theory Bienenstock–Cooper–Munro (BCM) theory, BCM synaptic modification, or the BCM rule, named after Elie Bienenstock, Leon Cooper, and Paul Munro, is a physical theory of learning in the visual cortex developed in 1981. The BCM model proposes a sli ...

have seen some success in mathematically describing these networks. In the mammalian

central nervous system The central nervous system (CNS) is the part of the nervous system consisting primarily of the brain, spinal cord and retina. The CNS is so named because the brain integrates the received information and coordinates and influences the activity o ...

, signal transmission is carried out by interconnected networks of nerve cells, or neurons. For the basic

pyramidal neuron Pyramidal cells, or pyramidal neurons, are a type of multipolar neuron found in areas of the brain including the cerebral cortex, the hippocampus, and the amygdala. Pyramidal cells are the primary excitation units of the mammalian prefrontal cort ...

, the input signal is carried by the

axon An axon (from Greek ἄξων ''áxōn'', axis) or nerve fiber (or nerve fibre: see American and British English spelling differences#-re, -er, spelling differences) is a long, slender cellular extensions, projection of a nerve cell, or neuron, ...

, which releases neurotransmitter chemicals into the

synapse In the nervous system, a synapse is a structure that allows a neuron (or nerve cell) to pass an electrical or chemical signal to another neuron or a target effector cell. Synapses can be classified as either chemical or electrical, depending o ...

which is picked up by the

dendrites A dendrite (from Greek δένδρον ''déndron'', "tree") or dendron is a branched cytoplasmic process that extends from a nerve cell that propagates the electrochemical stimulation received from other neural cells to the cell body, or soma ...

of the next neuron, which can then generate an

action potential An action potential (also known as a nerve impulse or "spike" when in a neuron) is a series of quick changes in voltage across a cell membrane. An action potential occurs when the membrane potential of a specific Cell (biology), cell rapidly ri ...

which is analogous to the output signal in the computational case. The synaptic weight in this process is determined by several variable factors: * How well the input signal propagates through the axon (see

myelination Myelination, or myelinogenesis, is the formation and development of myelin sheaths in the nervous system, typically initiated in late prenatal neurodevelopment and continuing throughout postnatal development. The term ''myelinogenesis'' is also ...

), * The amount of neurotransmitter released into the synapse and the amount that can be absorbed in the following cell (determined by the number of

AMPA α-Amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid, better known as AMPA, is a compound that is a specific agonist for the AMPA receptor, where it mimics the effects of the neurotransmitter glutamate Glutamic acid (symbol Glu or E; kn ...

and

NMDA receptor The ''N''-methyl-D-aspartate receptor (also known as the NMDA receptor or NMDAR), is a glutamate receptor and predominantly Ca2+ ion channel found in neurons. The NMDA receptor is one of three types of ionotropic glutamate receptors, the other ...

s on the cell membrane and the amount of intracellular

calcium Calcium is a chemical element; it has symbol Ca and atomic number 20. As an alkaline earth metal, calcium is a reactive metal that forms a dark oxide-nitride layer when exposed to air. Its physical and chemical properties are most similar to it ...

and other ions), * The number of such connections made by the axon to the dendrites, * How well the signal propagates and integrates in the postsynaptic cell. The changes in synaptic weight that occur is known as

synaptic plasticity In neuroscience, synaptic plasticity is the ability of synapses to Chemical synapse#Synaptic strength, strengthen or weaken over time, in response to increases or decreases in their activity. Since memory, memories are postulated to be represent ...

, and the process behind long-term changes (

long-term potentiation In neuroscience, long-term potentiation (LTP) is a persistent strengthening of synapses based on recent patterns of activity. These are patterns of synaptic activity that produce a long-lasting increase in signal transmission between two neuron ...

and depression) is still poorly understood. Hebb's original learning rule was originally applied to biological systems, but has had to undergo many modifications as a number of theoretical and experimental problems came to light.

References

Artificial neural networks Neural circuitry Neuroplasticity {{comp-sci-stub

Computation

Biology

See also

References