In
theoretical computer science
Theoretical computer science (TCS) is a subset of general computer science and mathematics that focuses on mathematical aspects of computer science such as the theory of computation, lambda calculus, and type theory.
It is difficult to circumsc ...
, communication complexity studies the amount of communication required to solve a problem when the input to the problem is
distributed Distribution may refer to:
Mathematics
*Distribution (mathematics), generalized functions used to formulate solutions of partial differential equations
*Probability distribution, the probability of a particular value or value range of a varia ...
among two or more parties. The study of communication complexity was first introduced by
Andrew Yao
Andrew Chi-Chih Yao (; born December 24, 1946) is a Chinese computer scientist and computational theorist. He is currently a professor and the dean of Institute for Interdisciplinary Information Sciences (IIIS) at Tsinghua University. Yao us ...
in 1979, while studying the problem of computation distributed among several machines.
The problem is usually stated as follows: two parties (traditionally called
Alice and Bob
Alice and Bob are fictional characters commonly used as placeholders in discussions about cryptographic systems and protocols, and in other science and engineering literature where there are several participants in a thought experiment. The Al ...
) each receive a (potentially different)
-
bit
The bit is the most basic unit of information in computing and digital communications. The name is a portmanteau of binary digit. The bit represents a logical state with one of two possible values. These values are most commonly represented a ...
string
and
. The ''goal'' is for Alice to compute the value of a certain function,
, that depends on both
and
, with the least amount of
communication
Communication (from la, communicare, meaning "to share" or "to be in relation with") is usually defined as the transmission of information. The term may also refer to the message communicated through such transmissions or the field of inqu ...
between them.
While Alice and Bob can always succeed by having Bob send his whole
-bit string to Alice (who then computes the
function
Function or functionality may refer to:
Computing
* Function key, a type of key on computer keyboards
* Function model, a structured representation of processes in a system
* Function object or functor or functionoid, a concept of object-orie ...
), the idea here is to find clever ways of calculating ''
'' with fewer than
bits of communication. Note that, unlike in
computational complexity theory
In theoretical computer science and mathematics, computational complexity theory focuses on classifying computational problems according to their resource usage, and relating these classes to each other. A computational problem is a task solved ...
, communication complexity is not concerned with the
amount of computation performed by Alice or Bob, or the size of the
memory
Memory is the faculty of the mind by which data or information is encoded, stored, and retrieved when needed. It is the retention of information over time for the purpose of influencing future action. If past events could not be remembered ...
used, as we generally assume nothing about the computational power of either Alice or Bob.
This abstract problem with two parties (called two-party communication complexity), and its general form with
more than two parties, is relevant in many contexts. In
VLSI
Very large-scale integration (VLSI) is the process of creating an integrated circuit (IC) by combining millions or billions of MOS transistors onto a single chip. VLSI began in the 1970s when MOS integrated circuit (Metal Oxide Semiconductor) ...
circuit design, for example, one seeks to minimize energy used by decreasing the amount of electric signals passed between the different components during a distributed computation. The problem is also relevant in the study of data structures and in the optimization of computer networks. For surveys of the field, see the textbooks by and .
Formal definition
Let
where we assume in the typical case that
and
. Alice holds an
-bit string
while Bob holds an
-bit string
. By communicating to each other one
bit
The bit is the most basic unit of information in computing and digital communications. The name is a portmanteau of binary digit. The bit represents a logical state with one of two possible values. These values are most commonly represented a ...
at a time (adopting some
''communication protocol'' which is agreed upon in advance), Alice and Bob wish to compute the value of
such that at least one party knows the value at the end of the communication. At this point the answer can be communicated back so that at the cost of one extra bit, both parties will know the answer. The worst case communication complexity of this communication problem of computing
, denoted as
, is then defined to be
:
minimum number of bits exchanged between Alice and Bob in the worst case.
As observed above, for any function
, we have
.
Using the above definition, it is useful to think of the function
as a
matrix
Matrix most commonly refers to:
* ''The Matrix'' (franchise), an American media franchise
** '' The Matrix'', a 1999 science-fiction action film
** "The Matrix", a fictional setting, a virtual reality environment, within ''The Matrix'' (franchi ...
(called the ''input matrix'' or ''communication matrix'') where the rows are indexed by
and columns by
. The entries of the matrix are
. Initially both Alice and Bob have a copy of the entire matrix
(assuming the function
is known to both parties). Then, the problem of computing the function value can be rephrased as "zeroing-in" on the corresponding matrix entry. This problem can be solved if either Alice or Bob knows both
and
. At the start of communication, the number of choices for the value of the function on the inputs is the size of matrix, i.e.
. Then, as and when each party communicates a bit to the other, the number of choices for the answer reduces as this eliminates a set of rows/columns resulting in a
submatrix
In mathematics, a matrix (plural matrices) is a rectangle, rectangular array variable, array or table of numbers, symbol (formal), symbols, or expression (mathematics), expressions, arranged in rows and columns, which is used to represent a math ...
of
.
More formally, a set
is called a ''(combinatorial) rectangle'' if whenever
and
then
. Equivalently,
is a combinatorial rectangle if it can be expressed as
for some
and
. Consider the case when
bits are already exchanged between the parties. Now, for a particular
, let us define a matrix
:
Then,
, and it is not hard to show that
is a combinatorial rectangle in
.
Example:
We consider the case where Alice and Bob try to determine whether or not their input strings are equal. Formally, define the ''Equality'' function, denoted
, by
if
. As we demonstrate below, any deterministic communication protocol solving
requires
bits of communication in the worst case. As a warm-up example, consider the simple case of
. The equality function in this case can be represented by the matrix below. The rows represent all the possibilities of
, the columns those of
.
As you can see, the function only evaluates to 1 when
equals
(i.e., on the diagonal). It is also fairly easy to see how communicating a single bit divides your possibilities in half. If you know that the first bit of
is 1, you only need to consider half of the columns (where
can equal 100, 101, 110, or 111).
Theorem:
Proof. Assume that
. This means that there exists
such that
and
have the same communication transcript
. Since this transcript defines a rectangle,
must also be 1. By definition
and we know that equality is only true for
when
. This yields a contradiction.
This technique of proving deterministic communication lower bounds is called the ''fooling set'' technique.
Randomized communication complexity
In the above definition, we are concerned with the number of bits that must be ''deterministically'' transmitted between two parties. If both the parties are given access to a random number generator, can they determine the value of
with much less information exchanged? Yao, in his seminal paper
[
answers this question by defining randomized communication complexity.
A randomized protocol for a function has two-sided error.
:
:
A randomized protocol is a deterministic protocol that uses an extra random string in addition to its normal input. There are two models for this: a ''public string'' is a random string that is known by both parties beforehand, while a ''private string'' is generated by one party and must be communicated to the other party. A theorem presented below shows that any public string protocol can be simulated by a private string protocol that uses ''O(log n)'' additional bits compared to the original.
Note that in the probability inequalities above, the outcome of the protocol is understood to depend ''only'' on the random string; both strings ''x'' and ''y'' remain fixed. In other words, if R(x,y) yields g(x,y,r) when using random string ''r'', then g(x,y,r) = f(x,y) for at least 2/3 of all choices for the string ''r''.
The randomized complexity is simply defined as the number of bits exchanged in such a protocol.
Note that it is also possible to define a randomized protocol with one-sided error, and the complexity is defined similarly.
]
Example: EQ
Returning to the previous example of ''EQ'', if certainty is not required, Alice and Bob can check for equality using only messages. Consider the following protocol: Assume that Alice and Bob both have access to the same random string . Alice computes and sends this bit (call it ''b'') to Bob. (The is the dot product
In mathematics, the dot product or scalar productThe term ''scalar product'' means literally "product with a scalar as a result". It is also used sometimes for other symmetric bilinear forms, for example in a pseudo-Euclidean space. is an alg ...
in GF(2)
(also denoted \mathbb F_2, or \mathbb Z/2\mathbb Z) is the finite field of two elements (GF is the initialism of ''Galois field'', another name for finite fields). Notations and \mathbb Z_2 may be encountered although they can be confused with ...
.) Then Bob compares ''b'' to . If they are the same, then Bob accepts, saying ''x'' equals ''y''. Otherwise, he rejects.
Clearly, if , then , so . If ''x'' does not equal ''y'', it is still possible that , which would give Bob the wrong answer. How does this happen?
If ''x'' and ''y'' are not equal, they must differ in some locations:
:
Where and agree, so those terms affect the dot products equally. We can safely ignore those terms and look only at where and differ. Furthermore, we can swap the bits and without changing whether or not the dot products are equal. This means we can swap bits so that contains only zeros and contains only ones:
:
Note that and . Now, the question becomes: for some random string , what is the probability that ? Since each is equally likely to be or , this probability is just . Thus, when does not equal ,
. The algorithm can be repeated many times to increase its accuracy. This fits the requirements for a randomized communication algorithm.
This shows that ''if Alice and Bob share a random string of length n'', they can send one bit to each other to compute . In the next section, it is shown that Alice and Bob can exchange only bits that are as good as sharing a random string of length ''n''. Once that is shown, it follows that ''EQ'' can be computed in messages.
Example: GH
For yet another example of randomized communication complexity, we turn to an example known as the ''gap-Hamming problem
In communication complexity, the gap-Hamming problem asks, if Alice and Bob are each given a (potentially different) string, what is the minimal number of bits that they need to exchange in order for Alice to approximately compute the Hamming dist ...
'' (abbreviated ''GH''). Formally, Alice and Bob both maintain binary messages, and would like to determine if the strings are very similar or if they are not very similar. In particular, they would like to find a communication protocol requiring the transmission of as few bits as possible to compute the following partial Boolean function,
:
Clearly, they must communicate all their bits if the protocol is to be deterministic (this is because, if there is a deterministic, strict subset of indices that Alice and Bob relay to one another, then imagine having a pair of strings that on that set disagree in positions. If another disagreement occurs in any position that is not relayed, then this affects the result of , and hence would result in an incorrect procedure.
A natural question one then asks is, if we're permitted to err of the time (over random instances drawn uniformly at random from ), then can we get away with a protocol with fewer bits? It turns out that the answer somewhat surprisingly is no, due to a result of Chakrabarti and Regev in 2012: they show that for random instances, any procedure which is correct at least of the time must send bits worth of communication, which is to say essentially all of them.
Public coins versus private coins
It is easier to create random protocols when both parties have access to the same random string (shared string protocol). It is still possible to use these protocols even when the two parties don't share a random string (private string protocol) with a small communication cost. Any shared string random protocol using any number of random string can be simulated by a private string protocol that uses an extra ''O(log n)'' bits.
Intuitively, we can find some set of strings that has enough randomness in it to run the random protocol with only a small increase in error. This set can be shared beforehand, and instead of drawing a random string, Alice and Bob need only agree on which string to choose from the shared set. This set is small enough that the choice can be communicated efficiently. A formal proof follows.
Consider some random protocol ''P'' with a maximum error rate of 0.1. Let be strings of length ''n'', numbered . Given such an , define a new protocol which randomly picks some and then runs ''P'' using as the shared random string. It takes ''O''(log 100''n'') = ''O''(log ''n'') bits to communicate the choice of .
Let us define and to be the probabilities that and compute the correct value for the input .
For a fixed , we can use Hoeffding's inequality
In probability theory, Hoeffding's inequality provides an upper bound on the probability that the sum of bounded independent random variables deviates from its expected value by more than a certain amount. Hoeffding's inequality was proven by Wass ...
to get the following equation:
: