HOME

TheInfoList



OR:

In
probability Probability is the branch of mathematics concerning numerical descriptions of how likely an event is to occur, or how likely it is that a proposition is true. The probability of an event is a number between 0 and 1, where, roughly speakin ...
and
statistics Statistics (from German: ''Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, industri ...
, memorylessness is a property of certain
probability distribution In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon ...
s. It usually refers to the cases when the distribution of a "waiting time" until a certain event does not depend on how much time has elapsed already. To model memoryless situations accurately, we must constantly 'forget' which state the system is in: the probabilities would not be influenced by the history of the process. Only two kinds of distributions are memoryless:
geometric distribution In probability theory and statistics, the geometric distribution is either one of two discrete probability distributions: * The probability distribution of the number ''X'' of Bernoulli trials needed to get one success, supported on the set \; * ...
s of non-negative integers and the
exponential distribution In probability theory and statistics, the exponential distribution is the probability distribution of the time between events in a Poisson point process, i.e., a process in which events occur continuously and independently at a constant average ...
s of non-negative real numbers. In the context of
Markov process A Markov chain or Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought of as, "What happe ...
es, memorylessness refers to the
Markov property In probability theory and statistics, the term Markov property refers to the memoryless property of a stochastic process. It is named after the Russian mathematician Andrey Markov. The term strong Markov property is similar to the Markov proper ...
, an even stronger assumption which implies that the properties of random variables related to the future depend only on relevant information about the current time, not on information from further in the past. The present article describes the use outside the Markov property.


Waiting time examples


With memory

Most phenomena are not memoryless, which means that observers will obtain information about them over time. For example, suppose that is a
random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...
, the lifetime of a car engine, expressed in terms of "number of miles driven until the engine breaks down". It is clear, based on our intuition, that an engine which has already been driven for 300,000 miles will have a much lower than would a second (equivalent) engine which has only been driven for 1,000 miles. Hence, this random variable would not have the memorylessness property.


Without memory

In contrast, let us examine a situation which would exhibit memorylessness. Imagine a long hallway, lined on one wall with thousands of safes. Each safe has a dial with 500 positions, and each has been assigned an opening position at random. Imagine that an eccentric person walks down the hallway, stopping once at each safe to make a single random attempt to open it. In this case, we might define random variable as the lifetime of their search, expressed in terms of "number of attempts the person must make until they successfully open a safe". In this case, will always be equal to the value of 500, regardless of how many attempts have already been made. Each new attempt has a (1/500) chance of succeeding, so the person is likely to open exactly one safe sometime in the next 500 attempts – but with each new failure they make no "progress" toward ultimately succeeding. Even if the safe-cracker has just failed 499 consecutive times (or 4,999 times), we expect to wait 500 more attempts until we observe the next success. If, instead, this person focused their attempts on a single safe, and "remembered" their previous attempts to open it, they would be guaranteed to open the safe after, at most, 500 attempts (and, in fact, at onset would only expect to need 250 attempts, not 500). Real-life examples of memorylessness include the universal law of radioactive decay, which describes the time until a given radioactive particle decays, and, potentially, the time until the discovery of a new Bitcoin block, though this has been put in question. An often used (theoretical) example of memorylessness in
queueing theory Queueing theory is the mathematical study of waiting lines, or queues. A queueing model is constructed so that queue lengths and waiting time can be predicted. Queueing theory is generally considered a branch of operations research because the ...
is the time a storekeeper must wait before the arrival of the next customer.


Discrete memorylessness

Suppose is a
discrete random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...
whose values lie in the set . The probability distribution of is memoryless precisely if for any and in , we have :\Pr(X>m+n \mid X \ge m)=\Pr(X>n). Here, denotes the
conditional probability In probability theory, conditional probability is a measure of the probability of an event occurring, given that another event (by assumption, presumption, assertion or evidence) has already occurred. This particular method relies on event B occur ...
that the value of is greater than given that it is greater than or equal to . The memoryless discrete probability distributions are the
geometric distribution In probability theory and statistics, the geometric distribution is either one of two discrete probability distributions: * The probability distribution of the number ''X'' of Bernoulli trials needed to get one success, supported on the set \; * ...
s, which count the number of
independent Independent or Independents may refer to: Arts, entertainment, and media Artist groups * Independents (artist group), a group of modernist painters based in the New Hope, Pennsylvania, area of the United States during the early 1930s * Independe ...
, identically distributed
Bernoulli trial In the theory of probability and statistics, a Bernoulli trial (or binomial trial) is a random experiment with exactly two possible outcomes, "success" and "failure", in which the probability of success is the same every time the experiment is c ...
s needed to get one "success". In other words, these are the distributions of waiting time in a Bernoulli process. Note that the above definition applies to the definition of
geometric distribution In probability theory and statistics, the geometric distribution is either one of two discrete probability distributions: * The probability distribution of the number ''X'' of Bernoulli trials needed to get one success, supported on the set \; * ...
with support . The alternative parameterization with support corresponds to a slightly different definition of discrete memorylessness: namely, that \Pr(X>m+n \mid X > m)=\Pr(X>n).


A common misunderstanding

"Memorylessness" of the probability distribution of the number of failures before the first success means that, for example, :\Pr(X>40 \mid X \ge 30)=\Pr(X>10). It does mean that :\Pr(X>40 \mid X \ge 30)=\Pr(X>40), which would be true only if the events and were independent, i.e. \Pr(X \ge 30)=1.


Continuous memorylessness

Suppose is a continuous random variable whose values lie in the non-negative real numbers . The probability distribution of is memoryless precisely if for any non-negative
real number In mathematics, a real number is a number that can be used to measure a ''continuous'' one-dimensional quantity such as a distance, duration or temperature. Here, ''continuous'' means that values can have arbitrarily small variations. Every r ...
s and , we have :\Pr(X>t+s \mid X>t)=\Pr(X>s). This is similar to the discrete version, except that and are constrained only to be non-negative real numbers instead of
integer An integer is the number zero (), a positive natural number (, , , etc.) or a negative integer with a minus sign (−1, −2, −3, etc.). The negative numbers are the additive inverses of the corresponding positive numbers. In the language o ...
s. Rather than counting trials until the first "success", for example, we may be marking time until the arrival of the first phone call at a switchboard.


The memoryless distribution is an exponential distribution

The only memoryless continuous probability distribution is the
exponential distribution In probability theory and statistics, the exponential distribution is the probability distribution of the time between events in a Poisson point process, i.e., a process in which events occur continuously and independently at a constant average ...
, so memorylessness completely characterizes the exponential distribution among all continuous ones. The property is derived through the following proof: To see this, first define the
survival function The survival function is a function that gives the probability that a patient, device, or other object of interest will survive past a certain time. The survival function is also known as the survivor function or reliability function. The te ...
, , as :S(t) = \Pr(X > t). Note that is then
monotonically decreasing In mathematics, a monotonic function (or monotone function) is a function between ordered sets that preserves or reverses the given order. This concept first arose in calculus, and was later generalized to the more abstract setting of ord ...
. From the relation :\Pr(X > t + s \mid X > t) = \Pr(X > s) and the definition of
conditional probability In probability theory, conditional probability is a measure of the probability of an event occurring, given that another event (by assumption, presumption, assertion or evidence) has already occurred. This particular method relies on event B occur ...
, it follows that :\frac = \Pr(X > s). This gives the
functional equation In mathematics, a functional equation is, in the broadest meaning, an equation in which one or several functions appear as unknowns. So, differential equations and integral equations are functional equations. However, a more restricted meaning ...
(which is a result of the memorylessness property): :S(t + s) = S(t) S(s) From this, we must have for example: :S(2) = S(1)^2 \quad :S(1) = S(1/2)^2 \text\quad S(1/2) = S(1)^. In general: :S(a) = S(1)^a The only continuous function that will satisfy this equation for any positive, rational is: :S(a) = S(1)^a = e^ = e^, where \lambda = - \ln(S(1)). Therefore, since is a probability and must have \lambda >0 , then any memorylessness function must be an exponential. Put a different way, is a monotone decreasing function (meaning that for times x\leq y, then S(x)\geq S(y).) The functional equation alone will imply that restricted to
rational Rationality is the quality of being guided by or based on reasons. In this regard, a person acts rationally if they have a good reason for what they do or a belief is rational if it is based on strong evidence. This quality can apply to an abili ...
multiples of any particular number is an
exponential function The exponential function is a mathematical function denoted by f(x)=\exp(x) or e^x (where the argument is written as an exponent). Unless otherwise specified, the term generally refers to the positive-valued function of a real variable, a ...
. Combined with the fact that is monotone, this implies that over its whole domain is an exponential function.


Notes


References

* Feller, W. (1971) ''Introduction to Probability Theory and Its Applications, Vol II'' (2nd edition),Wiley. Section I.3 {{isbn, 0-471-25709-5 Theory of probability distributions Characterization of probability distributions Markov processes