A committee machine is a type of
artificial neural network
Artificial neural networks (ANNs), usually simply called neural networks (NNs) or neural nets, are computing systems inspired by the biological neural networks that constitute animal brains.
An ANN is based on a collection of connected units ...
using a
divide and conquer
Divide and rule policy ( la, divide et impera), or divide and conquer, in politics and sociology is gaining and maintaining power divisively. Historically, this strategy was used in many different ways by empires seeking to expand their terr ...
strategy in which the responses of multiple neural networks (experts) are combined into a single response.
[HAYKIN, S. Neural Networks - A Comprehensive Foundation. Second edition. Pearson Prentice Hall: 1999.] The combined response of the committee machine is supposed to be superior to those of its constituent experts. Compare with
ensembles of classifiers.
Types
Static structures
In this class of committee machines, the responses of several predictors (experts) are combined by means of a mechanism that does not involve the input signal, hence the designation static. This category includes the following methods:
*
Ensemble averaging
In machine learning, particularly in the creation of artificial neural networks, ensemble averaging is the process of creating multiple models and combining them to produce a desired output, as opposed to creating just one model. Frequently an ens ...
In ensemble averaging, outputs of different predictors are linearly combined to produce an overall output.
*
Boosting
In boosting, a weak algorithm is converted into one that achieves arbitrarily high accuracy.
Dynamic structures
In this second class of committee machines, the input signal is directly involved in actuating the mechanism that integrates the outputs of the individual experts into an overall output, hence the designation dynamic. There are two kinds of dynamic structures:
*
Mixture of experts Mixture of experts (MoE) refers to a machine learning technique where multiple expert networks (learners) are used to divide a problem space into homogeneous regions. It differs from ensemble techniques in that typically only a few, or 1, expert m ...
In mixture of experts, the individual responses of the experts are non-linearly combined by means of a single gating network.
*Hierarchical mixture of experts
In hierarchical mixture of experts, the individual responses of the individual experts are non-linearly combined by means of several gating networks arranged in a hierarchical fashion.
References
{{Reflist
Artificial neural networks
Learning