In
computational complexity theory
In theoretical computer science and mathematics, computational complexity theory focuses on classifying computational problems according to their resource usage, and relating these classes to each other. A computational problem is a task solved ...
, a complexity class is a
set of
computational problem
In theoretical computer science, a computational problem is a problem that may be solved by an algorithm. For example, the problem of factoring
:"Given a positive integer ''n'', find a nontrivial prime factor of ''n''."
is a computational probl ...
s of related resource-based
complexity. The two most commonly analyzed resources are
time
Time is the continued sequence of existence and events that occurs in an apparently irreversible succession from the past, through the present, into the future. It is a component quantity of various measurements used to sequence events, ...
and
memory
Memory is the faculty of the mind by which data or information is encoded, stored, and retrieved when needed. It is the retention of information over time for the purpose of influencing future action. If past events could not be remembered ...
.
In general, a complexity class is defined in terms of a type of computational problem, a
model of computation
In computer science, and more specifically in computability theory and computational complexity theory, a model of computation is a model which describes how an output of a mathematical function is computed given an input. A model describes how ...
, and a bounded resource like
time
Time is the continued sequence of existence and events that occurs in an apparently irreversible succession from the past, through the present, into the future. It is a component quantity of various measurements used to sequence events, ...
or
memory
Memory is the faculty of the mind by which data or information is encoded, stored, and retrieved when needed. It is the retention of information over time for the purpose of influencing future action. If past events could not be remembered ...
. In particular, most complexity classes consist of
decision problem
In computability theory and computational complexity theory, a decision problem is a computational problem that can be posed as a yes–no question of the input values. An example of a decision problem is deciding by means of an algorithm wheth ...
s that are solvable with a
Turing machine
A Turing machine is a mathematical model of computation describing an abstract machine that manipulates symbols on a strip of tape according to a table of rules. Despite the model's simplicity, it is capable of implementing any computer algori ...
, and are differentiated by their time or space (memory) requirements. For instance, the class
P is the set of decision problems solvable by a deterministic Turing machine in
polynomial time. There are, however, many complexity classes defined in terms of other types of problems (e.g.
counting problems and
function problem
In computational complexity theory, a function problem is a computational problem where a single output (of a total function) is expected for every input, but the output is more complex than that of a decision problem. For function problems, the o ...
s) and using other models of computation (e.g.
probabilistic Turing machine
In theoretical computer science, a probabilistic Turing machine is a non-deterministic Turing machine that chooses between the available transitions at each point according to some probability distribution. As a consequence, a probabilistic Turi ...
s,
interactive proof system
In computational complexity theory, an interactive proof system is an abstract machine that models computation as the exchange of messages between two parties: a ''prover'' and a ''verifier''. The parties interact by exchanging messages in order t ...
s,
Boolean circuit
In computational complexity theory and circuit complexity, a Boolean circuit is a mathematical model for combinational digital logic circuits. A formal language can be decided by a family of Boolean circuits, one circuit for each possible inp ...
s, and
quantum computers).
The study of the relationships between complexity classes is a major area of research in theoretical computer science. There are often general hierarchies of complexity classes; for example, it is known that a number of fundamental time and space complexity classes relate to each other in the following way:
NL⊆
P⊆
NP⊆
PSPACE
In computational complexity theory, PSPACE is the set of all decision problems that can be solved by a Turing machine using a polynomial amount of space.
Formal definition
If we denote by SPACE(''t''(''n'')), the set of all problems that can b ...
⊆
EXPTIME⊆
EXPSPACE (where ⊆ denotes the
subset relation). However, many relationships are not yet known; for example, one of the most famous
open problems in computer science concerns whether
P equals NP. The relationships between classes often answer questions about the fundamental nature of computation. The P versus NP problem, for instance, is directly related to questions of whether
nondeterminism
Nondeterminism or nondeterministic may refer to:
Computer science
*Nondeterministic programming
*Nondeterministic algorithm
*Nondeterministic model of computation
**Nondeterministic finite automaton
**Nondeterministic Turing machine
*Indeterminacy ...
adds any computational power to computers and whether problems having solutions that can be quickly checked for correctness can also be quickly solved.
Background
Complexity classes are
sets of related
computational problem
In theoretical computer science, a computational problem is a problem that may be solved by an algorithm. For example, the problem of factoring
:"Given a positive integer ''n'', find a nontrivial prime factor of ''n''."
is a computational probl ...
s. They are defined in terms of the computational difficulty of solving the problems contained within them with respect to particular computational resources like time or memory. More formally, the definition of a complexity class consists of three things: a type of computational problem, a model of computation, and a bounded computational resource. In particular, most complexity classes consist of
decision problem
In computability theory and computational complexity theory, a decision problem is a computational problem that can be posed as a yes–no question of the input values. An example of a decision problem is deciding by means of an algorithm wheth ...
s that can be solved by a
Turing machine
A Turing machine is a mathematical model of computation describing an abstract machine that manipulates symbols on a strip of tape according to a table of rules. Despite the model's simplicity, it is capable of implementing any computer algori ...
with bounded
time
Time is the continued sequence of existence and events that occurs in an apparently irreversible succession from the past, through the present, into the future. It is a component quantity of various measurements used to sequence events, ...
or
space
Space is the boundless three-dimensional extent in which objects and events have relative position and direction. In classical physics, physical space is often conceived in three linear dimensions, although modern physicists usually cons ...
resources. For example, the complexity class
P is defined as the set of
decision problem
In computability theory and computational complexity theory, a decision problem is a computational problem that can be posed as a yes–no question of the input values. An example of a decision problem is deciding by means of an algorithm wheth ...
s that can be solved by a
deterministic Turing machine in
polynomial time.
Computational problems
Intuitively, a
computational problem
In theoretical computer science, a computational problem is a problem that may be solved by an algorithm. For example, the problem of factoring
:"Given a positive integer ''n'', find a nontrivial prime factor of ''n''."
is a computational probl ...
is just a question that can be solved by an
algorithm
In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific problems or to perform a computation. Algorithms are used as specifications for performing ...
. For example, "is the
natural number
In mathematics, the natural numbers are those numbers used for counting (as in "there are ''six'' coins on the table") and ordering (as in "this is the ''third'' largest city in the country").
Numbers used for counting are called ''cardinal ...
prime
A prime number (or a prime) is a natural number greater than 1 that is not a product of two smaller natural numbers. A natural number greater than 1 that is not prime is called a composite number. For example, 5 is prime because the only ways ...
?" is a computational problem. A computational problem is mathematically represented as the
set of answers to the problem. In the primality example, the problem (call it
) is represented by the set of all natural numbers that are prime:
. In the theory of computation, these answers are represented as
strings; for example, in the primality example the natural numbers could be represented as strings of
bit
The bit is the most basic unit of information in computing and digital communications. The name is a portmanteau of binary digit. The bit represents a logical state with one of two possible values. These values are most commonly represente ...
s that represent
binary numbers. For this reason, computational problems are often synonymously referred to as languages, since strings of bits represent
formal language
In logic, mathematics, computer science, and linguistics, a formal language consists of words whose letters are taken from an alphabet and are well-formed according to a specific set of rules.
The alphabet of a formal language consists of sy ...
s (a concept borrowed from
linguistics
Linguistics is the science, scientific study of human language. It is called a scientific study because it entails a comprehensive, systematic, objective, and precise analysis of all aspects of language, particularly its nature and structure ...
); for example, saying that the
problem is in the complexity class
NP is equivalent to saying that the language
is in NP.
Decision problems
The most commonly analyzed problems in theoretical computer science are
decision problem
In computability theory and computational complexity theory, a decision problem is a computational problem that can be posed as a yes–no question of the input values. An example of a decision problem is deciding by means of an algorithm wheth ...
s—the kinds of problems that can be posed as
yes-no questions. The primality example above, for instance, is an example of a decision problem as it can be represented by the yes-no question "is the
natural number
In mathematics, the natural numbers are those numbers used for counting (as in "there are ''six'' coins on the table") and ordering (as in "this is the ''third'' largest city in the country").
Numbers used for counting are called ''cardinal ...
prime
A prime number (or a prime) is a natural number greater than 1 that is not a product of two smaller natural numbers. A natural number greater than 1 that is not prime is called a composite number. For example, 5 is prime because the only ways ...
". In terms of the theory of computation, a decision problem is represented as the set of input strings that a computer running a correct
algorithm
In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific problems or to perform a computation. Algorithms are used as specifications for performing ...
would answer "yes" to. In the primality example,
is the set of strings representing natural numbers that, when input into a computer running an algorithm that correctly
tests for primality, the algorithm answers "yes, this number is prime". This "yes-no" format is often equivalently stated as "accept-reject"; that is, an algorithm "accepts" an input string if the answer to the decision problem is "yes" and "rejects" if the answer is "no".
While some problems cannot easily be expressed as decision problems, they nonetheless encompass a broad range of computational problems. Other types of problems that certain complexity classes are defined in terms of include
function problem
In computational complexity theory, a function problem is a computational problem where a single output (of a total function) is expected for every input, but the output is more complex than that of a decision problem. For function problems, the o ...
s (e.g.
FP),
counting problems (e.g.
#P),
optimization problem
In mathematics, computer science and economics, an optimization problem is the problem of finding the ''best'' solution from all feasible solutions.
Optimization problems can be divided into two categories, depending on whether the variables ...
s, and
promise problem
In computational complexity theory, a promise problem is a generalization of a decision problem where the input is promised to belong to a particular subset of all possible inputs. Unlike decision problems, the ''yes'' instances (the inputs for whi ...
s (see section "Other types of problems").
Computational models
To make concrete the notion of a "computer", in theoretical computer science problems are analyzed in the context of a
computational model
A computational model uses computer programs to simulate and study complex systems using an algorithmic or mechanistic approach and is widely used in a diverse range of fields spanning from physics, chemistry and biology to economics, psychology, ...
. Computational models make exact the notions of computational resources like "time" and "memory". In
computational complexity theory
In theoretical computer science and mathematics, computational complexity theory focuses on classifying computational problems according to their resource usage, and relating these classes to each other. A computational problem is a task solved ...
, complexity classes deal with the ''inherent'' resource requirements of problems and not the resource requirements that depend upon how a physical computer is constructed. For example, in the real world different computers may require different amounts of time and memory to solve the same problem because of the way that they have been engineered. By providing an abstract mathematical representations of computers, computational models abstract away superfluous complexities of the real world (like differences in
processor
Processor may refer to:
Computing Hardware
* Processor (computing)
**Central processing unit (CPU), the hardware within a computer that executes a program
*** Microprocessor, a central processing unit contained on a single integrated circuit (I ...
speed) that obstruct an understanding of fundamental principles.
The most commonly used computational model is the
Turing machine
A Turing machine is a mathematical model of computation describing an abstract machine that manipulates symbols on a strip of tape according to a table of rules. Despite the model's simplicity, it is capable of implementing any computer algori ...
. While other models exist and many complexity classes are defined in terms of them (see section
"Other models of computation"), the Turing machine is used to define most basic complexity classes. With the Turing machine, instead of using standard units of time like the second (which make it impossible to disentangle running time from the speed of physical hardware) and standard units of memory like
byte
The byte is a unit of digital information that most commonly consists of eight bits. Historically, the byte was the number of bits used to encode a single character of text in a computer and for this reason it is the smallest addressable uni ...
s, the notion of time is abstracted as the number of elementary steps that a Turing machine takes to solve a problem and the notion of memory is abstracted as the number of cells that are used on the machine's tape. These are explained in greater detail below.
It is also possible to use the
Blum axioms In computational complexity theory the Blum axioms or Blum complexity axioms are axioms that specify desirable properties of complexity measures on the set of computable functions. The axioms were first defined by Manuel Blum in 1967.
Importantly ...
to define complexity classes without referring to a concrete
computational model
A computational model uses computer programs to simulate and study complex systems using an algorithmic or mechanistic approach and is widely used in a diverse range of fields spanning from physics, chemistry and biology to economics, psychology, ...
, but this approach is less frequently used in complexity theory.
Deterministic Turing machines
A Turing machine is a mathematical model of a general computing machine. It is the most commonly used model in complexity theory, owing in large part to the fact that it is believed to be as powerful as any other model of computation and is easy to analyze mathematically. Importantly, it is believed that if there exists an algorithm that solves a particular problem then there also exists a Turing machine that solves that same problem (this is known as the
Church–Turing thesis); this means that it is believed that ''every'' algorithm can be represented as a Turing machine.
Mechanically, a Turing machine (TM) manipulates symbols (generally restricted to the bits 0 and 1 to provide an intuitive connection to real-life computers) contained on an infinitely long strip of tape. The TM can read and write, one at a time, using a tape head. Operation is fully determined by a finite set of elementary instructions such as "in state 42, if the symbol seen is 0, write a 1; if the symbol seen is 1, change into state 17; in state 17, if the symbol seen is 0, write a 1 and change to state 6". The Turing machine starts with only the input string on its tape and blanks everywhere else. The TM accepts the input if it enters a designated accept state and rejects the input if it enters a reject state. The deterministic Turing machine (DTM) is the most basic type of Turing machine. It uses a fixed set of rules to determine its future actions (which is why it is called "
deterministic").
A computational problem can then be defined in terms of a Turing machine as the set of input strings that a particular Turing machine accepts. For example, the primality problem
from above is the set of strings (representing natural numbers) that a Turing machine running an algorithm that correctly
tests for primality accepts. A Turing machine is said to recognize a language (recall that "problem" and "language" are largely synonymous in computability and complexity theory) if it accepts all inputs that are in the language and is said to decide a language if it additionally rejects all inputs that are not in the language (certain inputs may cause a Turing machine to run forever, so
decidability places the additional constraint over
recognizability that the Turing machine must halt on all inputs). A Turing machine that "solves" a problem is generally meant to mean one that decides the language.
Turing machines enable intuitive notions of "time" and "space". The
time complexity
In computer science, the time complexity is the computational complexity that describes the amount of computer time it takes to run an algorithm. Time complexity is commonly estimated by counting the number of elementary operations performed by t ...
of a TM on a particular input is the number of elementary steps that the Turing machine takes to reach either an accept or reject state. The
space complexity
The space complexity of an algorithm or a computer program is the amount of memory space required to solve an instance of the computational problem as a function of characteristics of the input. It is the memory required by an algorithm until it ex ...
is the number of cells on its tape that it uses to reach either an accept or reject state.
Nondeterministic Turing machines
The deterministic Turing machine (DTM) is a variant of the nondeterministic Turing machine (NTM). Intuitively, an NTM is just a regular Turing machine that has the added capability of being able to explore multiple possible future actions from a given state, and "choosing" a branch that accepts (if any accept). That is, while a DTM must follow only one branch of computation, an NTM can be imagined as a computation tree, branching into many possible computational pathways at each step (see image). If at least one branch of the tree halts with an "accept" condition, then the NTM accepts the input. In this way, an NTM can be thought of as simultaneously exploring all computational possibilities in parallel and selecting an accepting branch. NTMs are not meant to be physically realizable models, they are simply theoretically interesting abstract machines that give rise to a number of interesting complexity classes (which often do have physically realizable equivalent definitions).
The
time complexity
In computer science, the time complexity is the computational complexity that describes the amount of computer time it takes to run an algorithm. Time complexity is commonly estimated by counting the number of elementary operations performed by t ...
of an NTM is the maximum number of steps that the NTM uses on ''any'' branch of its computation. Similarly, the
space complexity
The space complexity of an algorithm or a computer program is the amount of memory space required to solve an instance of the computational problem as a function of characteristics of the input. It is the memory required by an algorithm until it ex ...
of an NTM is the maximum number of cells that the NTM uses on any branch of its computation.
DTMs can be viewed as a special case of NTMs that do not make use of the power of nondeterminism. Hence, every computation that can be carried out by a DTM can also be carried out by an equivalent NTM. It is also possible to simulate any NTM using a DTM (the DTM will simply compute every possible computational branch one-by-one). Hence, the two are equivalent in terms of computability. However, simulating an NTM with a DTM often requires greater time and/or memory resources; as will be seen, how significant this slowdown is for certain classes of computational problems is an important question in computational complexity theory.
Resource bounds
Complexity classes group computational problems by their resource requirements. To do this, computational problems are differentiated by
upper bound
In mathematics, particularly in order theory, an upper bound or majorant of a subset of some preordered set is an element of that is greater than or equal to every element of .
Dually, a lower bound or minorant of is defined to be an eleme ...
s on the maximum amount of resources that the most efficient algorithm takes to solve them. More specifically, complexity classes are concerned with the ''rate of growth'' in the resources required to solve particular computational problems as the input size increases. For example, the amount of time it takes to solve problems in the complexity class
P grows at a
polynomial
In mathematics, a polynomial is an expression consisting of indeterminates (also called variables) and coefficients, that involves only the operations of addition, subtraction, multiplication, and positive-integer powers of variables. An example ...
rate as the input size increases, which is comparatively slow compared to problems in the exponential complexity class
EXPTIME (or more accurately, for problems in EXPTIME that are outside of P, since
).
Note that the study of complexity classes is intended primarily to understand the ''inherent'' complexity required to solve computational problems. Complexity theorists are thus generally concerned with finding the smallest complexity class that a problem falls into and are therefore concerned with identifying which class a computational problem falls into using the ''most efficient'' algorithm. There may be an algorithm, for instance, that solves a particular problem in exponential time, but if the most efficient algorithm for solving this problem runs in polynomial time then the inherent time complexity of that problem is better described as polynomial.
Time bounds
The
time complexity
In computer science, the time complexity is the computational complexity that describes the amount of computer time it takes to run an algorithm. Time complexity is commonly estimated by counting the number of elementary operations performed by t ...
of an algorithm with respect to the Turing machine model is the number of steps it takes for a Turing machine to run an algorithm on a given input size. Formally, the time complexity for an algorithm implemented with a Turing machine
is defined as the function
, where
is the maximum number of steps that
takes on any input of length
.
In computational complexity theory, theoretical computer scientists are concerned less with particular runtime values and more with the general class of functions that the time complexity function falls into. For instance, is the time complexity function a
polynomial
In mathematics, a polynomial is an expression consisting of indeterminates (also called variables) and coefficients, that involves only the operations of addition, subtraction, multiplication, and positive-integer powers of variables. An example ...
? A
logarithmic function
In mathematics, the logarithm is the inverse function to exponentiation. That means the logarithm of a number to the base is the exponent to which must be raised, to produce . For example, since , the ''logarithm base'' 10 ...
? An
exponential function
The exponential function is a mathematical function denoted by f(x)=\exp(x) or e^x (where the argument is written as an exponent). Unless otherwise specified, the term generally refers to the positive-valued function of a real variable, ...
? Or another kind of function?
Space bounds
The
space complexity
The space complexity of an algorithm or a computer program is the amount of memory space required to solve an instance of the computational problem as a function of characteristics of the input. It is the memory required by an algorithm until it ex ...
of an algorithm with respect to the Turing machine model is the number of cells on the Turing machine's tape that are required to run an algorithm on a given input size. Formally, the space complexity of an algorithm implemented with a Turing machine
is defined as the function
, where
is the maximum number of cells that
uses on any input of length
.
Basic complexity classes
Basic definitions
Complexity classes are often defined using granular sets of complexity classes called DTIME and NTIME (for time complexity) and DSPACE and NSPACE (for space complexity). Using
big O notation, they are defined as follows:
* The time complexity class
is the set of all problems that are decided by an
time deterministic Turing machine.
* The time complexity class
is the set of all problems that are decided by an
time nondeterministic Turing machine.
* The space complexity class
is the set of all problems that are decided by an
space deterministic Turing machine.
* The space complexity class
is the set of all problems that are decided by an
space nondeterministic Turing machine.
Time complexity classes
P and NP
P is the class of problems that are solvable by a
deterministic Turing machine in
polynomial time and NP is the class of problems that are solvable by a
nondeterministic Turing machine
In theoretical computer science, a nondeterministic Turing machine (NTM) is a theoretical model of computation whose governing rules specify more than one possible action when in some given situations. That is, an NTM's next state is ''not'' comp ...
in polynomial time. Or more formally,
:
:
P is often said to be the class of problems that can be solved "quickly" or "efficiently" by a deterministic computer, since the
time complexity
In computer science, the time complexity is the computational complexity that describes the amount of computer time it takes to run an algorithm. Time complexity is commonly estimated by counting the number of elementary operations performed by t ...
of solving a problem in P increases relatively slowly with the input size.
An important characteristic of the class NP is that it can be equivalently defined as the class of problems whose solutions are ''verifiable'' by a deterministic Turing machine in polynomial time. That is, a language is in NP if there exists a ''deterministic'' polynomial time Turing machine, referred to as the verifier, that takes as input a string
''and'' a polynomial-size
certificate string
, and accepts
if
is in the language and rejects
if
is not in the language. Intuitively, the certificate acts as a
proof that the input
is in the language. Formally:
: NP is the class of languages
for which there exists a polynomial-time deterministic Turing machine
and a polynomial
such that for all
,
is in
''if and only if'' there exists some
such that
accepts.
This equivalence between the nondeterministic definition and the verifier definition highlights a fundamental connection between
nondeterminism
Nondeterminism or nondeterministic may refer to:
Computer science
*Nondeterministic programming
*Nondeterministic algorithm
*Nondeterministic model of computation
**Nondeterministic finite automaton
**Nondeterministic Turing machine
*Indeterminacy ...
and solution verifiability. Furthermore, it also provides a useful method for proving that a language is in NP—simply identify a suitable certificate and show that it can be verified in polynomial time.
=The P versus NP problem
=
While there might seem to be an obvious difference between the class of problems that are efficiently solvable and the class of problems whose solutions are merely efficiently checkable, P and NP are actually at the center of one of the most famous unsolved problems in computer science: the
P versus NP problem. While it is known that P
NP (intuitively, deterministic Turing machines are just a subclass of nondeterministic Turing machines that don't make use of their nondeterminism; or under the verifier definition, P is the class of problems whose polynomial time verifiers need only receive the empty string as their certificate), it is not known whether NP is strictly larger than P. If P=NP, then it follows that nondeterminism provides ''no additional computational power'' over determinism with regards to the ability to quickly find a solution to a problem; that is, being able to explore ''all possible branches'' of computation provides ''at most'' a polynomial speedup over being able to explore only a single branch. Furthermore, it would follow that if there exists a proof for a problem instance and that proof can be quickly be checked for correctness (that is, if the problem is in NP), then there also exists an algorithm that can quickly ''construct'' that proof (that is, the problem is in P). However, the overwhelming majority of computer scientists believe that P
NP, and most
cryptographic schemes employed today rely on the assumption that P
NP.
EXPTIME and NEXPTIME
EXPTIME (sometimes shortened to EXP) is the class of decision problems solvable by a deterministic Turing machine in exponential time and NEXPTIME (sometimes shortened to NEXP) is the class of decision problems solvable by a nondeterministic Turing machine in exponential time. Or more formally,
:
:
EXPTIME is a strict superset of P and NEXPTIME is a strict superset of NP. It is further the case that EXPTIME
NEXPTIME. It is not known whether this is proper, but if P=NP then EXPTIME must equal NEXPTIME.
Space complexity classes
L and NL
While it is possible to define
logarithmic time complexity classes, these are extremely narrow classes as sublinear times do not even enable a Turing machine to read the entire input (because
). However, there are a meaningful number of problems that can be solved in logarithmic space. The definitions of these classes require a
two-tape Turing machine so that it is possible for the machine to store the entire input (it can be shown that in terms of
computability
Computability is the ability to solve a problem in an effective manner. It is a key topic of the field of computability theory within mathematical logic and the theory of computation within computer science. The computability of a problem is clo ...
the two-tape Turing machine is equivalent to the single-tape Turing machine). In the two-tape Turing machine model, one tape is the input tape, which is read-only. The other is the work tape, which allows both reading and writing and is the tape on which the Turing machine performs computations. The space complexity of the Turing machine is measured as the number of cells that are used on the work tape.
L (sometimes lengthened to LOGSPACE) is then defined as the class of problems solvable in logarithmic space on a deterministic Turing machine and NL (sometimes lengthened to NLOGSPACE) is the class of problems solvable in logarithmic space on a nondeterministic Turing machine. Or more formally,
:
:
It is known that
. However, it is not known whether any of these relationships is proper.
PSPACE and NPSPACE
The complexity classes PSPACE and NPSPACE are the space analogues to
P and
NP. That is, PSPACE is the class of problems solvable in polynomial space by a deterministic Turing machine and NPSPACE is the class of problems solvable in polynomial space by a nondeterministic Turing machine. More formally,
:
:
While it is not known whether P=NP,
Savitch's theorem
In computational complexity theory, Savitch's theorem, proved by Walter Savitch in 1970, gives a relationship between deterministic and non-deterministic space complexity. It states that for any function f\in\Omega(\log(n)),
:\mathsf\left(f\lef ...
famously showed that PSPACE=NPSPACE. It is also known that P
PSPACE, which follows intuitively from the fact that, since writing to a cell on a Turing machine's tape is defined as taking one unit of time, a Turing machine operating in polynomial time can only write to polynomially many cells. It is suspected that P is strictly smaller than PSPACE, but this has not been proven.
EXPSPACE and NEXPSPACE
The complexity classes EXPSPACE and NEXPSPACE are the space analogues to
EXPTIME and
NEXPTIME In computational complexity theory, the complexity class NEXPTIME (sometimes called NEXP) is the set of decision problems that can be solved by a non-deterministic Turing machine using time 2^.
In terms of NTIME,
:\mathsf = \bigcup_ \mathsf(2^)
A ...
. That is, EXPSPACE is the class of problems solvable in exponential space by a deterministic Turing machine and NEXPSPACE is the class of problems solvable in exponential space by a nondeterministic Turing machine. Or more formally,
:
:
Savitch's theorem showed that EXPSPACE=NEXPSPACE. This class is extremely broad: it is known to be a strict superset of PSPACE, NP, and P, and is believed to be a strict superset of EXPTIME.
Properties of complexity classes
Closure
Complexity classes have a variety of
closure properties. For example, decision classes may be closed under
negation,
disjunction
In logic, disjunction is a logical connective typically notated as \lor and read aloud as "or". For instance, the English language sentence "it is raining or it is snowing" can be represented in logic using the disjunctive formula R \lor S ...
,
conjunction
Conjunction may refer to:
* Conjunction (grammar), a part of speech
* Logical conjunction, a mathematical operator
** Conjunction introduction, a rule of inference of propositional logic
* Conjunction (astronomy), in which two astronomical bodies ...
, or even under all
Boolean operations. Moreover, they might also be closed under a variety of quantification schemes. P, for instance, is closed under all Boolean operations, and under quantification over polynomially sized domains. Closure properties can be helpful in separating classes—one possible route to separating two complexity classes is to find some closure property possessed by one class but not by the other.
Each class X that is not closed under negation has a complement class co-X, which consists of the complements of the languages contained in X (i.e. co-X=
).
co-NP
In computational complexity theory, co-NP is a complexity class. A decision problem X is a member of co-NP if and only if its complement is in the complexity class NP. The class can be defined as follows: a decision problem is in co-NP precisely ...
, for instance, is one important complement complexity class, and sits at the center of the unsolved problem over whether co-NP=NP.
Closure properties are one of the key reasons many complexity classes are defined in the way that they are. Take, for example, a problem that can be solved in
time (that is, in linear time) and one that can be solved in, at best,
time. Both of these problems are in P, yet the runtime of the second grows considerably faster than the runtime of the first as the input size increases. One might ask whether it would be better to define the class of "efficiently solvable" problems using some smaller polynomial bound, like
, rather than all polynomials, which allows for such large discrepancies. It turns out, however, that the set of all polynomials is the smallest class of functions containing the linear functions that is also closed under addition, multiplication, and composition (for instance,
, which is a polynomial but
). Since we would like composing one efficient algorithm with another efficient algorithm to still be considered efficient, the polynomials are the smallest class that ensures composition of "efficient algorithms". (Note that the definition of P is also useful because, empirically, almost all problems in P that are practically useful do in fact have low order polynomial runtimes, and almost all problems outside of P that are practically useful do not have any known algorithms with small exponential runtimes, i.e. with
runtimes where
is close to 1.)
Reductions
Many complexity classes are defined using the concept of a reduction. A reduction is a transformation of one problem into another problem, i.e. a reduction takes inputs from one problem and transforms them into inputs of another problem. For instance, you can reduce ordinary base-10 addition
to base-2 addition by transforming
and
to their base-2 notation (e.g. 5+7 becomes 101+111). Formally, a problem
reduces to a problem
if there exists a function
such that for every
,
''if and only if''
.
Generally, reductions are used to capture the notion of a problem being at least as difficult as another problem. Thus we are generally interested in using a polynomial-time reduction, since any problem
that can be efficiently reduced to another problem
is no more difficult than
. Formally, a problem
is polynomial-time reducible to a problem
if there exists a ''polynomial-time'' computable function
such that for all
,
''if and only if''
.
Note that reductions can be defined in many different ways. Common reductions are
Cook reductions,
Karp reductions and
Levin reductions, and can vary based on resource bounds, such as
polynomial-time reduction
In computational complexity theory, a polynomial-time reduction is a method for solving one problem using another. One shows that if a hypothetical subroutine solving the second problem exists, then the first problem can be solved by transforming ...
s and
log-space reduction
In computational complexity theory, a log-space reduction is a reduction computable by a deterministic Turing machine using logarithmic space. Conceptually, this means it can keep a constant number of pointers into the input, along with a logari ...
s.
Hardness
Reductions motivate the concept of a problem being hard for a complexity class. A problem
is hard for a class of problems C if every problem in C can be polynomial-time reduced to
. Thus no problem in C is harder than
, since an algorithm for
allows us to solve any problem in C with at most polynomial slowdown. Of particular importance, the set of problems that are hard for NP is called the set of
NP-hard problems.
Completeness
If a problem
is hard for C and is also in C, then
is said to be
complete for C. This means that
is the hardest problem in C (since there could be many problems that are equally hard, more precisely
is as hard as the hardest problems in C).
Of particular importance is the class of
NP-complete problems—the most difficult problems in NP. Because all problems in NP can be polynomial-time reduced to NP-complete problems, finding an NP-complete problem that can be solved in polynomial time would mean that P = NP.
Relationships between complexity classes
Savitch's theorem
Savitch's theorem establishes the relationship between deterministic and nondetermistic space resources. It shows that if a nondeterministic Turing machine can solve a problem using
space, then a deterministic Turing machine can solve the same problem in
space, i.e. in the square of the space. Formally, Savitch's theorem states that for any
,
:
Important corollaries of Savitch's theorem are that PSPACE = NPSPACE (since the square of a polynomial is still a polynomial) and EXPSPACE = NEXPSPACE (since the square of an exponential is still an exponential).
These relationships answer fundamental questions about the power of nondeterminism compared to determinism. Specifically, Savitch's theorem shows that any problem that a nondeterministic Turing machine can solve in polynomial space, a deterministic Turing machine can also solve in polynomial space. Similarly, any problem that a nondeterministic Turing machine can solve in exponential space, a deterministic Turing machine can also solve in exponential space.
Hierarchy theorems
By definition of DTIME, it follows that
is contained in
if
, since
if
. However, this definition gives no indication of whether this inclusion is strict. For time and space requirements, the conditions under which the inclusion is strict are given by the time and space hierarchy theorems, respectively. They are called hierarchy theorems because they induce a proper hierarchy on the classes defined by constraining the respective resources. The hierarchy theorems enable one to make quantitative statements about how much more additional time or space is needed in order to increase the number of problems that can be solved.
The
time hierarchy theorem states that
:
.
The
space hierarchy theorem In computational complexity theory, the space hierarchy theorems are separation results that show that both deterministic and nondeterministic machines can solve more problems in (asymptotically) more space, subject to certain conditions. For exampl ...
states that
:
.
The time and space hierarchy theorems form the basis for most separation results of complexity classes. For instance, the time hierarchy theorem establishes that P is strictly contained in EXPTIME, and the space hierarchy theorem establishes that L is strictly contained in PSPACE.
Other models of computation
While deterministic and non-deterministic
Turing machine
A Turing machine is a mathematical model of computation describing an abstract machine that manipulates symbols on a strip of tape according to a table of rules. Despite the model's simplicity, it is capable of implementing any computer algori ...
s are the most commonly used models of computation, many complexity classes are defined in terms of other computational models. In particular,
* A number of classes are defined using
probabilistic Turing machine
In theoretical computer science, a probabilistic Turing machine is a non-deterministic Turing machine that chooses between the available transitions at each point according to some probability distribution. As a consequence, a probabilistic Turi ...
s, including the classes
BPP
BPP may refer to:
Education
* BPP Holdings, a holding company based in the United Kingdom
* BPP Law School, a law school based in the United Kingdom and a constituent school of BPP University
* BPP University, a private university based in the ...
,
PP,
RP, and
ZPP
* A number of classes are defined using
interactive proof system
In computational complexity theory, an interactive proof system is an abstract machine that models computation as the exchange of messages between two parties: a ''prover'' and a ''verifier''. The parties interact by exchanging messages in order t ...
s, including the classes
IP,
MA, and
AM
* A number of classes are defined using
Boolean circuit
In computational complexity theory and circuit complexity, a Boolean circuit is a mathematical model for combinational digital logic circuits. A formal language can be decided by a family of Boolean circuits, one circuit for each possible inp ...
s, including the classes
P/poly
In computational complexity theory, P/poly is a complexity class representing problems that can be solved by small circuits. More precisely, it is the set of formal languages that have polynomial-size circuit families. It can also be defined equiva ...
and its subclasses
NC and
AC
* A number of classes are defined using
quantum Turing machine
A quantum Turing machine (QTM) or universal quantum computer is an abstract machine used to model the effects of a quantum computer. It provides a simple model that captures all of the power of quantum computation—that is, any quantum algori ...
s, including the classes
BQP and
QMA
These are explained in greater detail below.
Randomized computation
A number of important complexity classes are defined using the
probabilistic Turing machine
In theoretical computer science, a probabilistic Turing machine is a non-deterministic Turing machine that chooses between the available transitions at each point according to some probability distribution. As a consequence, a probabilistic Turi ...
, a variant of the
Turing machine
A Turing machine is a mathematical model of computation describing an abstract machine that manipulates symbols on a strip of tape according to a table of rules. Despite the model's simplicity, it is capable of implementing any computer algori ...
that can toss random coins. These classes help to better describe the complexity of
randomized algorithms.
A probabilistic Turing machine is similar to a deterministic Turing machine, except rather than following a single transition function (a set of rules for how to proceed at each step of the computation) it probabilistically selects between multiple transition functions at each step. The standard definition of a probabilistic Turing machine specifies two transition functions, so that the selection of transition function at each step resembles a coin flip. The randomness introduced at each step of the computation introduces the potential for error; that is, strings that the Turing machine is meant to accept may on some occasions be rejected and strings that the Turing machine is meant to reject may on some occasions be accepted. As a result, the complexity classes based on the probabilistic Turing machine are defined in large part around the amount of error that is allowed. Formally, they are defined using an error probability
. A probabilistic Turing machine
is said to recognize a language
with error probability
if:
# a string
in
implies that
# a string
not in
implies that
Important complexity classes
The fundamental randomized time complexity classes are
ZPP,
RP,
co-RP,
BPP
BPP may refer to:
Education
* BPP Holdings, a holding company based in the United Kingdom
* BPP Law School, a law school based in the United Kingdom and a constituent school of BPP University
* BPP University, a private university based in the ...
, and
PP.
The strictest class is
ZPP (zero-error probabilistic polynomial time), the class of problems solvable in polynomial time by a probabilistic Turing machine with error probability 0. Intuitively, this is the strictest class of probabilistic problems because it demands ''no error whatsoever''.
A slightly looser class is
RP (randomized polynomial time), which maintains no error for strings not in the language but allows bounded error for strings in the language. More formally, a language is in RP if there is a probabilistic polynomial-time Turing machine
such that if a string is not in the language then
always rejects and if a string is in the language then
accepts with a probability at least 1/2. The class
co-RP is similarly defined except the roles are flipped: error is not allowed for strings in the language but is allowed for strings not in the language. Taken together, the classes RP and co-RP encompass all of the problems that can be solved by probabilistic Turing machines with
one-sided error
In computing, a Monte Carlo algorithm is a randomized algorithm whose output may be incorrect with a certain (typically small) probability. Two examples of such algorithms are Karger–Stein algorithm and Monte Carlo algorithm for minimum Feedba ...
.
Loosening the error requirements further to allow for
two-sided error
In computing, a Monte Carlo algorithm is a randomized algorithm whose output may be incorrect with a certain (typically small) probability. Two examples of such algorithms are Karger's algorithm, Karger–Stein algorithm and Monte Carlo algorithm ...
yields the class
BPP
BPP may refer to:
Education
* BPP Holdings, a holding company based in the United Kingdom
* BPP Law School, a law school based in the United Kingdom and a constituent school of BPP University
* BPP University, a private university based in the ...
(bounded-error probabilistic polynomial time), the class of problems solvable in polynomial time by a probabilistic Turing machine with error probability less than 1/3 (for both strings in the language and not in the language). BPP is the most practically relevant of the probabilistic complexity classes—problems in BPP have efficient
randomized algorithms that can be run quickly on real computers. BPP is also at the center of the important unsolved problem in computer science over whether
P=BPP, which if true would mean that randomness does not increase the computational power of computers, i.e. any probabilistic Turing machine could be simulated by a deterministic Turing machine with at most polynomial slowdown.
The broadest class of efficiently-solvable probabilistic problems is
PP (probabilistic polynomial time), the set of languages solvable by a probabilistic Turing machine in polynomial time with an error probability of less than 1/2 for all strings.
ZPP, RP and co-RP are all subsets of BPP, which in turn is a subset of PP. The reason for this is intuitive: the classes allowing zero error and only one-sided error are all contained within the class that allows two-sided error, and PP simply relaxes the error probability of BPP. ZPP relates to RP and co-RP in the following way:
. That is, ZPP consists exactly of those problems that are in both RP and co-RP. Intuitively, this follows from the fact that RP and co-RP allow only one-sided error: co-RP does not allow error for strings in the language and RP does not allow error for strings not in the language. Hence, if a problem is in both RP and co-RP, then there must be no error for strings both in ''and'' not in the language (i.e. no error whatsoever), which is exactly the definition of ZPP.
Important randomized space complexity classes include
BPL,
RL, and
RLP.
Interactive proof systems
A number of complexity classes are defined using
interactive proof systems
In computational complexity theory, an interactive proof system is an abstract machine that models computation as the exchange of messages between two parties: a ''prover'' and a ''verifier''. The parties interact by exchanging messages in order t ...
. Interactive proofs generalize the proofs definition of the complexity class
NP and yield insights into
cryptography
Cryptography, or cryptology (from grc, , translit=kryptós "hidden, secret"; and ''graphein'', "to write", or ''-logia'', "study", respectively), is the practice and study of techniques for secure communication in the presence of adver ...
,
approximation algorithms, and
formal verification
In the context of hardware and software systems, formal verification is the act of proving or disproving the correctness of intended algorithms underlying a system with respect to a certain formal specification or property, using formal met ...
.
Interactive proof systems are
abstract machine
An abstract machine is a computer science theoretical model that allows for a detailed and precise analysis of how a computer system functions. It is analogous to a mathematical function in that it receives inputs and produces outputs based on pr ...
s that model computation as the exchange of messages between two parties: a prover
and a verifier
. The parties interact by exchanging messages, and an input string is accepted by the system if the verifier decides to accept the input on the basis of the messages it has received from the prover. The prover
has unlimited computational power while the verifier has bounded computational power (the standard definition of interactive proof systems defines the verifier to be polynomially-time bounded). The prover, however, is untrustworthy (this prevents all languages from being trivially recognized by the proof system by having the computationally unbounded prover solve for whether a string is in a language and then sending a trustworthy "YES" or "NO" to the verifier), so the verifier must conduct an "interrogation" of the prover by "asking it" successive rounds of questions, accepting only if it develops a high degree of confidence that the string is in the language.
Important complexity classes
The class
NP is a simple proof system in which the verifier is restricted to being a deterministic polynomial-time
Turing machine
A Turing machine is a mathematical model of computation describing an abstract machine that manipulates symbols on a strip of tape according to a table of rules. Despite the model's simplicity, it is capable of implementing any computer algori ...
and the procedure is restricted to one round (that is, the prover sends only a single, full proof—typically referred to as the
certificate—to the verifier). Put another way, in the definition of the class NP (the set of decision problems for which the problem instances, when the answer is "YES", have proofs verifiable in polynomial time by a deterministic Turing machine) is a proof system in which the proof is constructed by an unmentioned prover and the deterministic Turing machine is the verifier. For this reason, NP can also be called dIP (deterministic interactive proof), though it is rarely referred to as such.
It turns out that NP captures the full power of interactive proof systems with deterministic (polynomial-time) verifiers because it can be shown that for any proof system with a deterministic verifier it is never necessary to need more than a single round of messaging between the prover and the verifier. Interactive proof systems that provide greater computational power over standard complexity classes thus require ''probabilistic'' verifiers, which means that the verifier's questions to the prover are computed using
probabilistic algorithm
A randomized algorithm is an algorithm that employs a degree of randomness as part of its logic or procedure. The algorithm typically uses uniformly random bits as an auxiliary input to guide its behavior, in the hope of achieving good performan ...
s. As noted in the section above on
randomized computation, probabilistic algorithms introduce error into the system, so complexity classes based on probabilistic proof systems are defined in terms of an error probability
.
The most general complexity class arising out of this characterization is the class
IP (interactive polynomial time), which is the class of all problems solvable by an interactive proof system
, where
is probabilistic polynomial-time and the proof system satisfies two properties: for a language
# (Completeness) a string
in
implies
# (Soundness) a string
not in
implies
An important feature of IP is that it equals
PSPACE
In computational complexity theory, PSPACE is the set of all decision problems that can be solved by a Turing machine using a polynomial amount of space.
Formal definition
If we denote by SPACE(''t''(''n'')), the set of all problems that can b ...
. In other words, any problem that can be solved by a polynomial-time interactive proof system can also be solved by a
deterministic Turing machine with polynomial space resources, and vice versa.
A modification of the protocol for IP produces another important complexity class:
AM (Arthur–Merlin protocol). In the definition of interactive proof systems used by IP, the prover was not able to see the coins utilized by the verifier in its probabilistic computation—it was only able to see the messages that the verifier produced with these coins. For this reason, the coins are called ''private random coins''. The interactive proof system can be constrained so that the coins used by the verifier are ''public random coins''; that is, the prover is able to see the coins. Formally, AM is defined as the class of languages with an interactive proof in which the verifier sends a random string to the prover, the prover responds with a message, and the verifier either accepts or rejects by applying a deterministic polynomial-time function to the message from the prover. AM can be generalized to AM
'k'' where ''k'' is the number of messages exchanged (so in the generalized form the standard AM defined above is AM
. However, it is the case that for all
, AM
'k''AM
It is also the case that