HOME

TheInfoList



OR:

The prosecutor's fallacy is a fallacy of statistical reasoning involving a test for an occurrence, such as a DNA match. A positive result in the test may
paradox A paradox is a logically self-contradictory statement or a statement that runs contrary to one's expectation. It is a statement that, despite apparently valid reasoning from true premises, leads to a seemingly self-contradictory or a logically u ...
ically be more likely to be an erroneous result than an actual occurrence, even if the test is very accurate. The fallacy is named because it is typically used by a prosecutor to exaggerate the probability of a criminal defendant's guilt. The fallacy can be used to support other claims as well – including the innocence of a defendant. For instance, if a perpetrator were known to have the same blood type as a given defendant and 10% of the population to share that blood type, then one version of the prosecutor's fallacy would be to claim that, on that basis alone, the probability that the defendant is guilty is 90%. However, this conclusion is only close to correct if the defendant was selected as the main suspect based on robust evidence discovered prior to the blood test and unrelated to it (the blood match may then be an "unexpected coincidence"). Otherwise, the reasoning presented is flawed, as it overlooks the high prior probability (that is, prior to the blood test) that he is a random innocent person. Assume, for instance, that 1000 people live in the town where the murder occurred. This means that 100 people live there who have the perpetrator's blood type; therefore, the true probability that the defendant is guilty – based on the fact that his blood type matches that of the killer – is only 1%, far less than the 90% argued by the prosecutor. At its heart, therefore, the fallacy involves assuming that the prior probability of a random match is equal to the probability that the defendant is innocent. When using it, a prosecutor questioning an expert witness may ask: "The odds of finding this evidence on an innocent man are so small that the jury can safely disregard the possibility that this defendant is innocent, correct?" The claim assumes that the probability that evidence is found on an innocent man is the same as the probability that a man is innocent given that evidence was found on him, which is not true. Whilst the former is usually small (approximately 10% in the previous example) due to good forensic evidence procedures, the latter (99% in that example) does not directly relate to it and will often be much higher, since, in fact, it depends on the likely quite high prior odds of the defendant being a random innocent person. Mathematically, the fallacy results from misunderstanding the concept of a
conditional probability In probability theory, conditional probability is a measure of the probability of an event occurring, given that another event (by assumption, presumption, assertion or evidence) has already occurred. This particular method relies on event B occur ...
, which is defined as the probability that an event ''A'' occurs given that event ''B'' is known – or assumed – to have occurred, and it is written as ''P(A, B)''. The error is based on assuming that ''P(A, B) = P(B, A)'', where ''A'' represents the event of finding evidence on the defendant, and ''B'' the event that the defendant is innocent. But this equality is not true: in fact, although ''P(A, B)'' is usually very small, ''P(B, A)'' may still be much higher.


Concept

The terms "prosecutor's fallacy" and "defense attorney's fallacy" were originated by William C. Thompson and Edward Schumann in 1987. The fallacy can arise from ''multiple testing'', such as when evidence is compared against a large database. The size of the database elevates the likelihood of finding a match by pure chance alone; i.e., DNA evidence is soundest when a match is found after a single directed comparison because the existence of matches against a large database where the test sample is of poor quality may be less unlikely by mere chance. The basic fallacy results from misunderstanding
conditional probability In probability theory, conditional probability is a measure of the probability of an event occurring, given that another event (by assumption, presumption, assertion or evidence) has already occurred. This particular method relies on event B occur ...
and neglecting the prior odds of a defendant being guilty before that evidence was introduced. When a prosecutor has collected some evidence (for instance a DNA match) and has an expert testify that the probability of finding this evidence if the accused were innocent is tiny, the fallacy occurs if it is concluded that the probability of the accused being innocent must be comparably tiny. If the DNA match is used to confirm guilt which is otherwise suspected then it is indeed strong evidence. However, if the DNA evidence is the sole evidence against the accused and the accused was picked out of a large database of DNA profiles, the odds of the match being made at random may be increased, and less damaging to the defendant. The odds in this scenario do not relate to the odds of being guilty, they relate to the odds of being picked at random. While the odds of being picked at random may be low for an individual condition implying guilt, i.e. a positive DNA match, the probability of being picked at random for ''any'' condition grows to 1 as more conditions are considered, as is the case in multiple testing. It is often the case that both innocence and guilt (i.e., accidental death and murder) are both highly improbable, though naturally one must be true, so the ratio of the likelihood of the "innocent scenario" to the "guilty scenario" is much more informative than the probability of the "guilty scenario" alone.


Examples


Conditional probability

In the fallacy of argument from rarity, an explanation for an observed event is said to be unlikely because the prior probability of that explanation is low. Consider this case: a lottery winner is accused of
cheating Cheating generally describes various actions designed to subvert rules in order to obtain unfair advantages. This includes acts of bribery, cronyism and nepotism in any situation where individuals are given preference using inappropriate crit ...
, based on the improbability of winning. At the trial, the prosecutor calculates the (very small) probability of winning the lottery without cheating and argues that this is the chance of innocence. The logical flaw is that the prosecutor has failed to account for the large number of people who play the lottery. While the probability of any singular person winning is quite low, the probability of any person winning the lottery, given the number of people who play it, is very high. In
Berkson's paradox Berkson's paradox, also known as Berkson's bias, collider bias, or Berkson's fallacy, is a result in conditional probability and statistics which is often found to be counterintuitive, and hence a veridical paradox. It is a complicating factor ar ...
,
conditional probability In probability theory, conditional probability is a measure of the probability of an event occurring, given that another event (by assumption, presumption, assertion or evidence) has already occurred. This particular method relies on event B occur ...
is mistaken for unconditional probability. This has led to several wrongful convictions of British mothers, accused of murdering two of their children in infancy, where the primary evidence against them was the statistical improbability of two children dying accidentally in the same household (under " Meadow's law"). Though multiple accidental (
SIDS Sudden infant death syndrome (SIDS) is the sudden unexplained death of a child of less than one year of age. Diagnosis requires that the death remain unexplained even after a thorough autopsy and detailed death scene investigation. SIDS usuall ...
) deaths are rare, so are multiple murders; with only the facts of the deaths as evidence, it is the ratio of these (prior) improbabilities that gives the correct "
posterior probability The posterior probability is a type of conditional probability that results from updating the prior probability with information summarized by the likelihood via an application of Bayes' rule. From an epistemological perspective, the posterior p ...
" of murder.


Multiple testing

In another scenario, a crime-scene
DNA sample Genetic testing, also known as DNA testing, is used to identify changes in DNA sequence or chromosome structure. Genetic testing can also include measuring the results of genetic changes, such as RNA analysis as an output of gene expression, or ...
is compared against a database of 20,000 men. A match is found, that man is accused and at his trial, it is testified that the probability that two DNA profiles match by chance is only 1 in 10,000. This does not mean the probability that the suspect is innocent is 1 in 10,000. Since 20,000 men were tested, there were 20,000 opportunities to find a match by chance. Even if none of the men in the database left the crime-scene DNA, a match by chance to an innocent is more likely than not. The chance of getting at least one match among the records is: : 1 - \left(1-\frac\right)^ \approx 86\%, :where, explicitly: ::1/10000 = probability of two DNA profiles matching by chance, after one check, ::(1 - 1/10000) = probability of not matching, after one check, ::(1 - 1/10000)^ = probability of not matching, after 20,000 checks, and ::1 - (1 - 1/10000)^ = probability of matching, after 20,000 checks. So, this evidence alone is an uncompelling data dredging result. If the culprit ''were'' in the database then he and one or more other men would probably be matched; in either case, it would be a fallacy to ignore the number of records searched when weighing the evidence. "Cold hits" like this on DNA databanks are now understood to require careful presentation as trial evidence.


Mathematical analysis

Finding a person innocent or guilty can be viewed in mathematical terms as a form of
binary classification Binary classification is the task of classifying the elements of a set into two groups (each called ''class'') on the basis of a classification rule. Typical binary classification problems include: * Medical testing to determine if a patient has ...
. If ''E'' is the observed evidence, and ''I'' stands for "accused is innocent" then consider the
conditional probabilities In probability theory, conditional probability is a measure of the probability of an event occurring, given that another event (by assumption, presumption, assertion or evidence) has already occurred. This particular method relies on event B occu ...
: * ''P''(''E'', ''I'') is the probability that the "damning evidence" would be observed even when the accused is innocent (a "false positive"). * ''P''(''I'', ''E'') is the probability that the accused is innocent, despite the evidence ''E''. With forensic evidence, ''P''(''E'', ''I'') is tiny. The prosecutor wrongly concludes that ''P''(''I'', ''E'') is comparatively tiny. (The
Lucia de Berk Lucia de Berk (born September 22, 1961, in The Hague, Netherlands), often called Lucia de B., is a Dutch licensed paediatric nurse who was the subject of a miscarriage of justice. In 2003, she was sentenced to life imprisonment, for which no paro ...
prosecution is accused of exactly this error, for example.) In fact, ''P''(''E'', ''I'') and ''P''(''I'', ''E'') are quite different; using
Bayes' theorem In probability theory and statistics, Bayes' theorem (alternatively Bayes' law or Bayes' rule), named after Thomas Bayes, describes the probability of an event, based on prior knowledge of conditions that might be related to the event. For examp ...
: : P(I, E) = P(E, I) \cdot \frac where: * ''P''(''I'') is the probability of innocence ''independent of the test result'' (i.e. from all other evidence) and * ''P''(''E'') is the prior probability that the evidence would be observed (regardless of innocence). This equation shows that a small P(E, I) does not imply a small P(I, E) in case of a large P(I) and a small P(E). That is, if the accused is otherwise likely to be innocent and it is unlikely for anyone (guilty or innocent) to exhibit the observed evidence. Note that : P(E) = P(E, I) \cdot P(I) + P(E, \sim I) \cdot - P(I) * ''P''(''E'', ~''I'') is the probability that the evidence would identify a guilty suspect (not give a
false negative A false positive is an error in binary classification in which a test result incorrectly indicates the presence of a condition (such as a disease when the disease is not present), while a false negative is the opposite error, where the test result ...
). This is usually close to 100%, slightly increasing the inference of innocence over a test with false negatives. That inequality is concisely expressed in terms of odds: : \operatorname(I, E) \ge \operatorname(I)\cdot P(E, I) The prosecutor is claiming a negligible chance of innocence, given the evidence, implying ''Odds''(''I'', ''E'') -> ''P''(''I'', ''E''), or that: : P(I, E) \approx P(E, I)\cdot \operatorname(I) A prosecutor conflating ''P''(''I'', ''E'') with ''P''(''E'', ''I'') makes a technical error whenever ''Odds''(''I'') ≫ 1. This may be a
harmless error In United States law, a harmless error is a ruling by a trial judge that, although mistaken, does not meet the burden for a losing party to reverse the original decision of the trier of fact on appeal, or to warrant a new trial. Harmless error i ...
if ''P''(''I'', ''E'') is still negligible, but it is especially misleading otherwise (mistaking low
statistical significance In statistical hypothesis testing, a result has statistical significance when it is very unlikely to have occurred given the null hypothesis (simply by chance alone). More precisely, a study's defined significance level, denoted by \alpha, is the p ...
for high
confidence Confidence is a state of being clear-headed either that a hypothesis or prediction is correct or that a chosen course of action is the best or most effective. Confidence comes from a Latin word 'fidere' which means "to trust"; therefore, having ...
).


Legal impact

Though the prosecutor's fallacy typically happens by mistake, in the
adversarial system The adversarial system or adversary system is a legal system used in the common law countries where two advocates represent their parties' case or position before an impartial person or group of people, usually a judge or jury, who attempt to det ...
lawyers are usually free to present statistical evidence as best suits their case; retrials are more commonly the result of the prosecutor's fallacy in expert witness testimony or in the judge's summation.


Defense attorney's fallacy

Suppose there is a one-in-a-million chance of a match given that the accused is innocent. The prosecutor says this means there is only a one-in-a-million chance of innocence. But if everyone in a community of 10 million people is tested, one expects 10 matches even if all are innocent. The defense fallacy would be to reason that "10 matches were expected, so the accused is no more likely to be guilty than any of the other matches, thus the evidence suggests a 90% chance that the accused is innocent." and "As such, this evidence is irrelevant." The first part of the reasoning would be correct only in the case where there is no further evidence pointing to the defendant. On the second part, Thompson & Schumann wrote that the evidence should still be highly
relevant Relevant is something directly related, connected or pertinent to a topic; it may also mean something that is current. Relevant may also refer to: * Relevant operator, a concept in physics, see renormalization group * Relevant, Ain, a commune ...
because it "drastically narrows the group of people who are or could have been suspects, while failing to exclude the defendant" (page 171). Another way of saying this would be to point out that the defense attorney's calculation failed to take into account the prior probability of the defendant's guilt. If, for example, the police came up with a list of 10 suspects, all of whom had access to the crime scene, then it would be very illogical indeed to suggest that a test that offers a one-in-a-million chance of a match would change the defendant's prior probability from 1 in 10 (10 percent) to 1 in a million (0.0001 percent). If nine innocent people were to be tested, the likelihood that the test would ''incorrectly'' match one (or more) of those people can be calculated as
:1 - ((1 - 1/1000000)^9), or approximately 0.0009%. If, however, the other 9 suspects were tested and did ''not'' return a match, then the probability of the defendant's guilt has increased from the prior probability of 10% (1 in 10 suspects) to 99.9991% on the basis of the test. The defendant might argue that "lists of suspects compiled by police fail to include the guilty person in 50% of cases" — if that were true, then the defendant's guilt would have increased from the prior probability of 5% (50% of 10%) to 49.99955% on the basis of the test — in which case " reasonable doubt" could be claimed to exist despite the positive test result.


Possible examples of fallacious defense arguments

Authors have cited defense arguments in the
O. J. Simpson murder trial ''The People of the State of California v. Orenthal James Simpson'' was a criminal trial in Los Angeles County Superior Court starting in 1994, in which O. J. Simpson, a former National Football League (NFL) player, broadcaster and actor, was tr ...
as an example of this fallacy regarding the context in which the accused had been brought to court: crime scene blood matched
Simpson Simpson most often refers to: * Simpson (name), a British surname *''The Simpsons'', an animated American sitcom **The Simpson family, central characters of the series ''The Simpsons'' Simpson may also refer to: Organizations Schools * Simp ...
's with characteristics shared by 1 in 400 people. The defense argued that a football stadium could be filled with Angelenos matching the sample and that the figure of 1 in 400 was useless. Also at the O. J. Simpson murder trial, the prosecution presented evidence that Simpson had been violent toward his wife, while the defense argued that there was only one woman murdered for every 2500 women who were subjected to spousal abuse, and that any history of Simpson being violent toward his wife was irrelevant to the trial. However, the reasoning behind the defense's calculation was fallacious. According to author Gerd Gigerenzer, the correct probability requires the context — that Simpson's wife had not only been subjected to domestic violence, but rather subjected to domestic violence (by Simpson) ''and'' killed (by someone) — to be taken into account. Gigerenzer writes "the chances that a batterer actually murdered his partner, given that she has been killed, is about 8 in 9 or approximately 90%". While most cases of spousal abuse do not end in murder, most cases of murder where there is a history of spousal abuse were committed by the spouse.


The Sally Clark case

Sally Clark Sally Clark (August 1964 – 15 March 2007) was an English solicitor who, in November 1999, became the victim of a miscarriage of justice when she was found guilty of the murder of her two infant sons. Clark's first son died in December 1996 wit ...
, a British woman, was accused in 1998 of having killed her first child at 11 weeks of age and then her second child at 8 weeks of age. The prosecution had expert witness Sir Roy Meadow, a professor and consultant paediatrician, testify that the probability of two children in the same family dying from
SIDS Sudden infant death syndrome (SIDS) is the sudden unexplained death of a child of less than one year of age. Diagnosis requires that the death remain unexplained even after a thorough autopsy and detailed death scene investigation. SIDS usuall ...
is about 1 in 73 million. That was much less frequent than the actual rate measured in historical data – Meadow estimated it from single-SIDS death data, and the assumption that the probability of such deaths should be
uncorrelated In probability theory and statistics, two real-valued random variables, X, Y, are said to be uncorrelated if their covariance, \operatorname ,Y= \operatorname Y- \operatorname \operatorname /math>, is zero. If two variables are uncorrelated, ther ...
between infants. Meadow acknowledged that 1-in-73 million is not an impossibility, but argued that such accidents would happen "once every hundred years" and that, in a country of 15 million 2-child families, it is vastly more likely that the double-deaths are due to
Münchausen syndrome by proxy Factitious disorder imposed on another (FDIA), also known as fabricated or induced illness by carers (FII), and first named as Munchausen syndrome by proxy (MSbP), is a condition in which a caregiver creates the appearance of health problems in a ...
than to such a rare accident. However, there is good reason to suppose that the likelihood of a death from SIDS in a family is significantly greater if a previous child has already died in these circumstances (a genetic predisposition to SIDS is likely to invalidate that assumed
statistical independence Independence is a fundamental notion in probability theory, as in statistics and the theory of stochastic processes. Two events are independent, statistically independent, or stochastically independent if, informally speaking, the occurrence of ...
) making some families more susceptible to SIDS and the error an outcome of the
ecological fallacy An ecological fallacy (also ecological ''inference'' fallacy or population fallacy) is a formal fallacy in the interpretation of statistical data that occurs when inferences about the nature of individuals are deduced from inferences about the gro ...
. The likelihood of two SIDS deaths in the same family cannot be soundly
estimated Estimation (or estimating) is the process of finding an estimate or approximation, which is a value that is usable for some purpose even if input data may be incomplete, uncertain, or unstable. The value is nonetheless usable because it is der ...
by squaring the likelihood of a single such death in all otherwise similar families. 1-in-73 million greatly underestimated the chance of two successive accidents, but, even if that assessment were accurate, the court seems to have missed the fact that the 1-in-73 million number meant nothing on its own. As an ''
a priori ("from the earlier") and ("from the later") are Latin phrases used in philosophy to distinguish types of knowledge, justification, or argument by their reliance on empirical evidence or experience. knowledge is independent from current ex ...
'' probability, it should have been weighed against the ''a priori'' probabilities of the alternatives. Given that two deaths had occurred, one of the following explanations must be true, and all of them are ''a priori'' extremely improbable: # Two successive deaths in the same family, both by SIDS # Double homicide (the prosecution's case) # Other possibilities (including one homicide and one case of SIDS) It's unclear whether an estimate of the probability for the second possibility was ever proposed during the trial, or whether the comparison of the first two probabilities was understood to be the key estimate to make in the statistical analysis assessing the prosecution's case against the case for innocence. Clark was convicted in 1999, resulting in a press release by the
Royal Statistical Society The Royal Statistical Society (RSS) is an established statistical society. It has three main roles: a British learned society for statistics, a professional body for statisticians and a charity which promotes statistics for the public good. ...
which pointed out the mistakes. In 2002, Ray Hill (Mathematics professor at
Salford Salford () is a city and the largest settlement in the City of Salford metropolitan borough in Greater Manchester, England. In 2011, Salford had a population of 103,886. It is also the second and only other city in the metropolitan county afte ...
) attempted to accurately compare the chances of these two possible explanations; he concluded that successive accidents are between 4.5 and 9 times more likely than are successive murders, so that the '"a priori"' odds of Clark's guilt were between 4.5 to 1 and 9 to 1 against. After it was found that the forensic pathologist who had examined both babies had withheld exculpatory evidence, a higher court later quashed Clark's conviction, on 29 January 2003.


See also

* *
Confusion of the inverse Confusion of the inverse, also called the conditional probability fallacy or the inverse fallacy, is a logical fallacy whereupon a conditional probability is equated with its inverse; that is, given two events ''A'' and ''B'', the probability of ''A ...
* * * * * * * Jurimetrics * * * *


References


External links


Discussion of the prosecutor's fallacy




{{DEFAULTSORT:Prosecutor's Fallacy Bayesian statistics Informal fallacies Forensic statistics Misuse of statistics Logic articles needing expert attention