__NOTOC__
In a
statistical-classification problem with two classes, a decision boundary or decision surface is a
hypersurface that partitions the underlying
vector space
In mathematics and physics, a vector space (also called a linear space) is a set (mathematics), set whose elements, often called vector (mathematics and physics), ''vectors'', can be added together and multiplied ("scaled") by numbers called sc ...
into two sets, one for each class. The classifier will classify all the points on one side of the decision boundary as belonging to one class and all those on the other side as belonging to the other class.
A decision boundary is the region of a problem space in which the output label of a
classifier is ambiguous.
If the decision surface is a
hyperplane
In geometry, a hyperplane is a generalization of a two-dimensional plane in three-dimensional space to mathematical spaces of arbitrary dimension. Like a plane in space, a hyperplane is a flat hypersurface, a subspace whose dimension is ...
, then the classification problem is linear, and the classes are
linearly separable.
Decision boundaries are not always clear cut. That is, the transition from one class in the
feature space
Feature may refer to:
Computing
* Feature recognition, could be a hole, pocket, or notch
* Feature (computer vision), could be an edge, corner or blob
* Feature (machine learning), in statistics: individual measurable properties of the phenom ...
to another is not discontinuous, but gradual. This effect is common in
fuzzy logic
Fuzzy logic is a form of many-valued logic in which the truth value of variables may be any real number between 0 and 1. It is employed to handle the concept of partial truth, where the truth value may range between completely true and completely ...
based classification algorithms, where membership in one class or another is ambiguous.
Decision boundaries can be approximations of optimal stopping boundaries. The decision boundary is the set of points of that hyperplane that pass through zero. For example, the angle between a vector and points in a set must be zero for points that are on or close to the decision boundary.
Decision boundary instability can be incorporated with generalization error as a standard for selecting the most accurate and stable classifier.
In neural networks and support vector models
In the case of
backpropagation based
artificial neural network
In machine learning, a neural network (also artificial neural network or neural net, abbreviated ANN or NN) is a computational model inspired by the structure and functions of biological neural networks.
A neural network consists of connected ...
s or
perceptrons, the type of decision boundary that the network can learn is determined by the number of hidden layers the network has.
If it has no hidden layers, then it can only learn linear problems. If it has one hidden layer, then it can learn any
continuous function
In mathematics, a continuous function is a function such that a small variation of the argument induces a small variation of the value of the function. This implies there are no abrupt changes in value, known as '' discontinuities''. More preci ...
on
compact subsets of
Rn as shown by the
universal approximation theorem, thus it can have an arbitrary decision boundary.
In particular,
support vector machines find a
hyperplane
In geometry, a hyperplane is a generalization of a two-dimensional plane in three-dimensional space to mathematical spaces of arbitrary dimension. Like a plane in space, a hyperplane is a flat hypersurface, a subspace whose dimension is ...
that separates the feature space into two classes with the
maximum margin. If the problem is not originally linearly separable, the
kernel trick can be used to turn it into a linearly separable one, by increasing the number of dimensions. Thus a general hypersurface in a small dimension space is turned into a hyperplane in a space with much larger dimensions.
Neural networks try to learn the decision boundary which minimizes the empirical error, while support vector machines try to learn the decision boundary which maximizes the empirical margin between the decision boundary and data points.
See also
*
Discriminant function
*
Hyperplane separation theorem
References
Further reading
*
{{DEFAULTSORT:Decision Boundary
Classification algorithms
Statistical classification
Pattern recognition#Probabilistic classifiers