Scoring algorithm

Scoring algorithm, also known as Fisher's scoring, is a form of Newton's method used in statistics to solve maximum likelihood equations numerically, named after Ronald Fisher.

Sketch of derivation

Let $Y_1,\ldots,Y_n$ be random variables, independent and identically distributed with twice differentiable p.d.f. $f(y; \theta)$, and we wish to calculate the maximum likelihood estimator (M.L.E.) $\theta^*$ of $\theta$. First, suppose we have a starting point for our algorithm $\theta_0$, and consider a Taylor expansion of the score function, $V(\theta)$, about $\theta_0$:

: $V(\theta) \approx V(\theta_0) - \mathcal(\theta_0)(\theta - \theta_0), \,$

where

: $\mathcal(\theta_0) = - \sum_^n \left. \nabla \nabla^ \_ \log f(Y_i ; \theta)$

is the observed information matrix at $\theta_0$. Now, setting $\theta = \theta^*$, using that $V(\theta^*) = 0$ and rearranging gives us:

: $\theta^* \approx \theta_ + \mathcal^(\theta_)V(\theta_). \,$

We therefore use the algorithm

: $\theta_ = \theta_ + \mathcal^(\theta_)V(\theta_), \,$

and under certain regularity conditions, it can be shown that $\theta_m \rightarrow \theta^*$.

Fisher scoring

In practice, $\mathcal(\theta)$ is usually replaced by \mathcal(\theta)= \mathrm mathcal(\theta)/math>, the Fisher information, thus giving us the Fisher Scoring Algorithm:

: $\theta_ = \theta_ + \mathcal^(\theta_)V(\theta_)$..

Under some regularity conditions, if $\theta_m$ is a consistent estimator, then $\theta_$ (the correction after a single step) is 'optimal' in the sense that its error distribution is asymptotically identical to that of the true max-likelihood estimate.

See also

*Score (statistics)
*Score test
*Fisher information

References

Further reading

*

{{Optimization algorithms, unconstrained

Maximum likelihood estimation