Optimal control theory is a branch of
mathematical optimization
Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. It is generally divided into two subfi ...
that deals with finding a
control
Control may refer to:
Basic meanings Economics and business
* Control (management), an element of management
* Control, an element of management accounting
* Comptroller (or controller), a senior financial officer in an organization
* Controlli ...
for a
dynamical system
In mathematics, a dynamical system is a system in which a function describes the time dependence of a point in an ambient space. Examples include the mathematical models that describe the swinging of a clock pendulum, the flow of water i ...
over a period of time such that an
objective function is optimized.
It has numerous applications in science, engineering and operations research. For example, the dynamical system might be a
spacecraft
A spacecraft is a vehicle or machine designed to fly in outer space. A type of artificial satellite, spacecraft are used for a variety of purposes, including communications, Earth observation, meteorology, navigation, space colonization, ...
with controls corresponding to rocket thrusters, and the objective might be to reach the
moon
The Moon is Earth's only natural satellite. It is the fifth largest satellite in the Solar System and the largest and most massive relative to its parent planet, with a diameter about one-quarter that of Earth (comparable to the width of ...
with minimum fuel expenditure. Or the dynamical system could be a nation's
economy
An economy is an area of the production, distribution and trade, as well as consumption of goods and services. In general, it is defined as a social domain that emphasize the practices, discourses, and material expressions associated with th ...
, with the objective to minimize
unemployment
Unemployment, according to the OECD (Organisation for Economic Co-operation and Development), is people above a specified age (usually 15) not being in paid employment or self-employment but currently available for work during the refe ...
; the controls in this case could be
fiscal and
monetary policy
Monetary policy is the policy adopted by the monetary authority of a nation to control either the interest rate payable for federal funds, very short-term borrowing (borrowing by banks from each other to meet their short-term needs) or the money s ...
. A dynamical system may also be introduced to embed
operations research problems within the framework of optimal control theory.
Optimal control is an extension of the
calculus of variations
The calculus of variations (or Variational Calculus) is a field of mathematical analysis that uses variations, which are small changes in functions
and functionals, to find maxima and minima of functionals: mappings from a set of functions t ...
, and is a
mathematical optimization
Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. It is generally divided into two subfi ...
method for deriving
control policies. The method is largely due to the work of
Lev Pontryagin
Lev Semenovich Pontryagin (russian: Лев Семёнович Понтрягин, also written Pontriagin or Pontrjagin) (3 September 1908 – 3 May 1988) was a Soviet mathematician. He was born in Moscow and lost his eyesight completely d ...
and
Richard Bellman
Richard Ernest Bellman (August 26, 1920 – March 19, 1984) was an American applied mathematician, who introduced dynamic programming in 1953, and made important contributions in other fields of mathematics, such as biomathematics. He founde ...
in the 1950s, after contributions to calculus of variations by
Edward J. McShane. Optimal control can be seen as a
control strategy
Control theory is a field of mathematics that deals with the control of dynamical systems in engineered processes and machines. The objective is to develop a model or algorithm governing the application of system inputs to drive the system to a ...
in
control theory
Control theory is a field of mathematics that deals with the control system, control of dynamical systems in engineered processes and machines. The objective is to develop a model or algorithm governing the application of system inputs to drive ...
.
General method
Optimal control deals with the problem of finding a control law for a given system such that a certain
optimality criterion is achieved. A control problem includes a
cost functional that is a
function of state and control variables. An optimal control is a set of
differential equation
In mathematics, a differential equation is an equation that relates one or more unknown functions and their derivatives. In applications, the functions generally represent physical quantities, the derivatives represent their rates of change, ...
s describing the paths of the control variables that minimize the cost function. The optimal control can be derived using
Pontryagin's maximum principle (a
necessary condition
In logic and mathematics, necessity and sufficiency are terms used to describe a conditional or implicational relationship between two statements. For example, in the conditional statement: "If then ", is necessary for , because the truth o ...
also known as Pontryagin's minimum principle or simply Pontryagin's principle), or by solving the
Hamilton–Jacobi–Bellman equation In optimal control theory, the Hamilton-Jacobi-Bellman (HJB) equation gives a necessary and sufficient condition for optimality of a control with respect to a loss function. It is, in general, a nonlinear partial differential equation in the val ...
(a
sufficient condition).
We begin with a simple example. Consider a car traveling in a straight line on a hilly road. The question is, how should the driver press the accelerator pedal in order to ''minimize'' the total traveling time? In this example, the term ''control law'' refers specifically to the way in which the driver presses the accelerator and shifts the gears. The ''system'' consists of both the car and the road, and the ''optimality criterion'' is the minimization of the total traveling time. Control problems usually include ancillary
constraints. For example, the amount of available fuel might be limited, the accelerator pedal cannot be pushed through the floor of the car, speed limits, etc.
A proper cost function will be a mathematical expression giving the traveling time as a function of the speed, geometrical considerations, and
initial conditions of the system.
Constraints are often interchangeable with the cost function.
Another related optimal control problem may be to find the way to drive the car so as to minimize its fuel consumption, given that it must complete a given course in a time not exceeding some amount. Yet another related control problem may be to minimize the total monetary cost of completing the trip, given assumed monetary prices for time and fuel.
A more abstract framework goes as follows.
Minimize the continuous-time cost functional
subject to the first-order dynamic constraints (the state equation)
the algebraic ''path constraints''
and the
endpoint conditions
where
is the ''state'',
is the ''control'',
is the independent variable (generally speaking, time),
is the initial time, and
is the terminal time. The terms
and
are called the ''endpoint cost '' and the ''running cost'' respectively. In the calculus of variations,
and
are referred to as the Mayer term and the ''
Lagrangian'', respectively. Furthermore, it is noted that the path constraints are in general ''inequality'' constraints and thus may not be active (i.e., equal to zero) at the optimal solution. It is also noted that the optimal control problem as stated above may have multiple solutions (i.e., the solution may not be unique). Thus, it is most often the case that any solution