Descent Direction

In optimization, a descent direction is a vector $\mathbf\in\mathbb R^n$ that, in the sense below, moves us closer towards a local minimum $\mathbf^*$ of our objective function $f:\mathbb R^n\to\mathbb R$.

Suppose we are computing $\mathbf^*$ by an iterative method, such as line search. We define a descent direction $\mathbf_k\in\mathbb R^n$ at the $k$th iterate to be any $\mathbf_k$ such that $\langle\mathbf_k,\nabla f(\mathbf_k)\rangle < 0$, where $\langle , \rangle$ denotes the inner product. The motivation for such an approach is that small steps along $\mathbf_k$ guarantee that $\displaystyle f$ is reduced, by Taylor's theorem.

Using this definition, the negative of a non-zero gradient is always a
descent direction, as $\langle -\nabla f(\mathbf_k), \nabla f(\mathbf_k) \rangle = -\langle \nabla f(\mathbf_k), \nabla f(\mathbf_k) \rangle < 0$.

Numerous methods exist to compute descent directions, all with differing merits. For example, one could use gradient descent or the conjugate gradient method.

More generally, if $P$ is a positive definite matrix, then
$p_k = -P \nabla f(x_k)$
is a descent direction at $x_k$. This generality is used in preconditioned gradient descent methods.

See also

* Directional derivative

References

{{DEFAULTSORT:Descent Direction
Mathematical optimization