Proximal Operator

In mathematical optimization, the proximal operator is an operator associated with a proper,An (extended) real-valued function ''f'' on a Hilbert space is said to be ''proper'' if it is not identically equal to $+\infty$, and $-\infty$ is not in its image. lower semi-continuous convex function $f$ from a Hilbert space $\mathcal$
to \infty,+\infty/math>, and is defined by:

::$\operatorname_f(v) = \arg \min_ \left(f(x) + \frac 1 2 \, x - v\, _\mathcal^2\right).$
For any function in this class, the minimizer of the right-hand side above is unique, hence making the proximal operator well-defined. The proximal operator is used in proximal gradient methods, which is frequently used in optimization algorithms associated with non-differentiable optimization problems such as total variation denoising.

Properties

The $\text$ of a proper, lower semi-continuous convex function $f$ enjoys several useful properties for optimization.

* Fixed points of $\text_f$ are minimizers of $f$: $\ = \arg \min f$.
* Global convergence to a minimizer is defined as follows: If $\arg \min f \neq \varnothing$, then for any initial point $x_0 \in \mathcal$, the recursion $(\forall n \in \mathbb)\quad x_ = \text_f x_n$ yields convergence $x_n \to x \in \arg \min f$ as $n \to +\infty$. This convergence may be weak if $\mathcal$ is infinite dimensional.
* The proximal operator can be seen as a generalization of the projection operator. Indeed, in the specific case where $f$ is the 0-$\infty$ characteristic function $\iota_C$ of a nonempty, closed, convex set $C$ we have that
: $\begin \operatorname_(x) &= \operatorname\limits_y \begin \frac \left\, x-y \right\, _2^2 & \text y \in C \\ + \infty & \text y \notin C \end \\ &=\operatorname\limits_ \frac \left\, x-y \right\, _2^2 \end$
: showing that the proximity operator is indeed a generalisation of the projection operator.

* A function is firmly non-expansive if $(\forall (x,y) \in \mathcal^2) \quad \, \text_fx - \text_fy\, ^2 \leq \langle x-y\ , \text_fx - \text_fy\rangle$.
* The proximal operator of a function is related to the gradient of the Moreau envelope $M_$ of a function $\lambda f$ by the following identity: $\nabla M_(x) = \frac (x - \mathrm_(x))$.

* The proximity operator of $f$ is characterized by inclusion $p=\operatorname_f(x) \Leftrightarrow x-p \in \partial f(p)$, where $\partial f$ is the subdifferential of $f$, given by
: $\partial f(x) = \$ In particular, If $f$ is differentiable then the above equation reduces to $p=\operatorname_f(x) \Leftrightarrow x-p = \nabla f(p)$.

Notes

References

{{Reflist

See also

* Proximal gradient method

External links

* Th
Proximity Operator repository
a collection of proximity operators implemented in Matlab and Python.

ProximalOperators.jl
a Julia package implementing proximal operators.

ODL
a Python library for inverse problems that utilizes proximal operators.
Mathematical optimization