In
machine learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence.
Machine ...
the margin of a single data point is defined to be the distance from the data point to a
decision boundary. Note that there are many distances and decision boundaries that may be appropriate for certain datasets and goals. A
margin classifier is a classifier that explicitly utilizes the margin of each example while learning a
classifier. There are theoretical justifications (based on the
VC dimension
VC may refer to:
Military decorations
* Victoria Cross, a military decoration awarded by the United Kingdom and also by certain Commonwealth nations
** Victoria Cross for Australia
** Victoria Cross (Canada)
** Victoria Cross for New Zealand
* ...
) as to why maximizing the margin (under some suitable constraints) may be beneficial for machine learning and statistical inferences algorithms.
There are many hyperplanes that might classify the data. One reasonable choice as the best hyperplane is the one that represents the largest separation, or margin, between the two classes. So we choose the hyperplane so that the distance from it to the nearest data point on each side is maximized. If such a hyperplane exists, it is known as the ''
maximum-margin hyperplane
In geometry, the hyperplane separation theorem is a theorem about disjoint convex sets in ''n''-dimensional Euclidean space. There are several rather similar versions. In one version of the theorem, if both these sets are closed and at least on ...
'' and the linear classifier it defines is known as a ''maximum
margin classifier''; or equivalently, the ''
perceptron
In machine learning, the perceptron (or McCulloch-Pitts neuron) is an algorithm for supervised classification, supervised learning of binary classification, binary classifiers. A binary classifier is a function which can decide whether or not an ...
of optimal stability.''
Support vector machines
{{compu-prog-stub