In
software engineering
Software engineering is a branch of both computer science and engineering focused on designing, developing, testing, and maintaining Application software, software applications. It involves applying engineering design process, engineering principl ...
and
mathematics
Mathematics is a field of study that discovers and organizes methods, Mathematical theory, theories and theorems that are developed and Mathematical proof, proved for the needs of empirical sciences and mathematics itself. There are many ar ...
, numerical error is the error in
the numerical computations.
Types
It can be the combined effect of two kinds of error in a calculation.
The first is referred to as
Round-off error
In computing, a roundoff error, also called rounding error, is the difference between the result produced by a given algorithm using exact arithmetic and the result produced by the same algorithm using finite-precision, rounded arithmetic. Roun ...
and is caused by the finite
precision of computations involving
floating-point
In computing, floating-point arithmetic (FP) is arithmetic on subsets of real numbers formed by a ''significand'' (a Sign (mathematics), signed sequence of a fixed number of digits in some Radix, base) multiplied by an integer power of that ba ...
numbers.
The second, usually called
Truncation error
In numerical analysis and scientific computing, truncation error is an error caused by approximating a mathematical process. The term truncation comes from the fact that these simplifications often involve the truncation of an infinite series expa ...
, is the difference between the exact mathematical solution and the approximate solution obtained when simplifications are made to the mathematical equations to make them more amenable to calculation.
Measure
Floating-point numerical error is often measured in ULP (
unit in the last place
In computer science and numerical analysis, unit in the last place or unit of least precision (ulp) is the spacing between two consecutive floating-point numbers, i.e., the value the '' least significant digit'' (rightmost digit) represents if it ...
).
See also
*
Loss of significance
*
Numerical analysis
Numerical analysis is the study of algorithms that use numerical approximation (as opposed to symbolic computation, symbolic manipulations) for the problems of mathematical analysis (as distinguished from discrete mathematics). It is the study of ...
*
Error analysis (mathematics) In mathematics, error analysis is the study of kind and quantity of error, or uncertainty, that may be present in the solution to a problem. This issue is particularly prominent in applied areas such as numerical analysis and statistics.
Error an ...
*
Round-off error
In computing, a roundoff error, also called rounding error, is the difference between the result produced by a given algorithm using exact arithmetic and the result produced by the same algorithm using finite-precision, rounded arithmetic. Roun ...
*
Kahan summation algorithm
*
Numerical sign problem
In applied mathematics, the numerical sign problem is the problem of numerically evaluating the integral of a highly oscillatory function of a large number of variables. Numerical methods fail because of the near-cancellation of the positive and n ...
References
* ''Accuracy and Stability of Numerical Algorithms'', Nicholas J. Higham,
* "Computational Error And Complexity In Science And Engineering", V. Lakshmikantham, S.K. Sen,
Computer arithmetic
Numerical analysis
{{applied-math-stub