Method
Let ''p'' = Pr(''X'' > ''Y''), and then test theAssumptions
Let ''Z''i = ''Y''i – ''X''i for ''i'' = 1, ... , ''n''. # The differences ''Zi'' are assumed to be independent. # Each ''Zi'' comes from the same continuous population. # The values ''X''''i'' and ''Y''''i'' represent are ordered (at least theSignificance testing
Since the test statistic is expected to follow aExample of two-sided sign test for matched pairs
Zar gives the following example of the sign test for matched pairs. Data are collected on the length of the left hind leg and left foreleg for 10 deer. The null hypothesis is that there is no difference between the hind leg and foreleg length in deer. The alternative hypothesis is that there is a difference between hind leg length and foreleg length. This is a two-tailed test, rather than a one-tailed test. For the two tailed test, the alternative hypothesis is that hind leg length may be either greater than or less than foreleg length. A one-sided test could be that hind leg length is greater than foreleg length, so that the difference can only be in one direction (greater than). There are n=10 deer. There are 8 positive differences and 2 negative differences. If the null hypothesis is true, that there is no difference in hind leg and foreleg lengths, then the expected number of positive differences is 5 out of 10. What is the probability that the observed result of 8 positive differences, or a more extreme result, would occur if there is no difference in leg lengths? Because the test is two-sided, a result as extreme or more extreme than 8 positive differences includes the results of 8, 9, or 10 positive differences, and the results of 0, 1, or 2 positive differences. The probability of 8 or more positives among 10 deer or 2 or fewer positives among 10 deer is the same as the probability of 8 or more heads or 2 or fewer heads in 10 flips of a fair coin. The probabilities can be calculated using theExamples
Example of one-sided sign test for matched pairs
Conover gives the following example using a one-sided sign test for matched pairs. A manufacturer produces two products, A and B. The manufacturer wishes to know if consumers prefer product B over product A. A sample of 10 consumers are each given product A and product B, and asked which product they prefer. The null hypothesis is that consumers do not prefer product B over product A. The alternative hypothesis is that consumers prefer product B over product A. This is a one-sided (directional) test. At the end of the study, 8 consumers preferred product B, 1 consumer preferred product A, and one reported no preference. *Number of +'s (preferred B) = 8 *Number of –'s (preferred A) = 1 *Number of ties (no preference) = 1 The tie is excluded from the analysis, giving n = number of +'s and –'s = 8 + 1 = 9. What is the probability of a result as extreme as 8 positives in favor of B in 9 pairs, if the null hypothesis is true, that consumers have no preference for B over A? This is the probability of 8 or more heads in 9 flips of a fair coin, and can be calculated using the binomial distribution with p(heads) = p(tails) = 0.5. P(8 or 9 heads in 9 flips of a fair coin) = 0.0195. The null hypothesis is rejected, and the manufacturer concludes that consumers prefer product B over product A.Example of sign test for median of a single sample
Sprent gives the following example of a sign test for a median. In a clinical trial, survival time (weeks) is collected for 10 subjects with non-Hodgkin's lymphoma. The exact survival time was not known for one subject who was still alive after 362 weeks, when the study ended. The subjects' survival times were : 49, 58, 75, 110, 112, 132, 151, 276, 281, 362+ The plus sign indicates the subject still alive at the end of the study. The researcher wished to determine if the median survival time was less than or greater than 200 weeks. The null hypothesis is that median survival is 200 weeks. The alternative hypothesis is that median survival is not 200 weeks. This is a two-sided test: the alternative median may be greater than or less than 200 weeks. If the null hypothesis is true, that the median survival is 200 weeks, then, in a random sample approximately half the subjects should survive less than 200 weeks, and half should survive more than 200 weeks. Observations below 200 are assigned a minus (−); observations above 200 are assigned a plus (+). For the subject survival times, there are 7 observations below 200 weeks (−) and 3 observations above 200 weeks (+) for the n=10 subjects. Because any one observation is equally likely to be above or below the population median, the number of plus scores will have a binomial distribution with mean = 0.5. What is the probability of a result as extreme as 7 in 10 subjects being below the median? This is exactly the same as the probability of a result as extreme as 7 heads in 10 tosses of a fair coin. Because this is a two-sided test, an extreme result can be either three or fewer heads or seven or more heads. The probability of observing k heads in 10 tosses of a fair coin, with p(heads) = 0.5, is given by the binomial formula: : Pr(Number of heads = ''k'') = Choose(10, ''k'') × 0.510 The probability for each value of ''k'' is given in the table below. The probability of 0, 1, 2, 3, 7, 8, 9, or 10 heads in 10 tosses is the sum of their individual probabilities: : 0.0010 + 0.0098 + 0.0439 + 0.1172 + 0.1172 + 0.0439 + 0.0098 + 0.0010 = 0.3438. Thus, the probability of observing 3 or fewer plus signs or 7 or more plus signs in the survival data, if the median survival is 200 weeks, is 0.3438. The expected number of plus signs is 5 if the null hypothesis is true. Observing 3 or fewer, or 7 or more pluses is not significantly different from 5. The null hypothesis is not rejected. Because of the extremely small sample size, this sample has low power to detect a difference.Software implementations
The sign test is a special case of the binomial test where the probability of success under the null hypothesis is p=0.5. Thus, the sign test can be performed using the binomial test, which is provided in most statistical software programs. On-line calculators for the sign test can be founded by searching for "sign test calculator". Many websites offer the binomial test, but generally offer only a two-sided version.Excel software for the sign test
A template for the sign test using Excel is available at http://www.real-statistics.com/non-parametric-tests/sign-test/R software for the sign test
In R, the binomial test can be performed using the functionbinom.test()
.
The syntax for the function is
x
= number of successes, or a vector of length 2 giving the numbers of successes and failures, respectively
*n
= number of trials; ignored if x has length 2
*p
= hypothesized probability of success
*alternative
=indicates the alternative hypothesis and must be one of "two.sided", "greater" or "less"
*conf.level
= confidence level for the returned confidence interval.
Examples of the sign test using the R function binom.test
The sign test example from Zar compared the length of hind legs and forelegs of deer. The hind leg was longer than the foreleg in 8 of 10 deer. Thus, there are x=8 successes in n=10 trials. The hypothesized probability of success (defined as hind leg longer than foreleg) is ''p'' = 0.5 under the null hypothesis that hind legs and forelegs do not differ in length. The alternative hypothesis is that hind leg length may be either greater than or less than foreleg length, which is a two sided test, specified as alternative="two.sided".
The R command History
Conover and Sprent describe John Arbuthnot's use of the sign test in 1710. Arbuthnot examined birth records in London for each of the 82 years from 1629 to 1710. In every year, the number of males born in London exceeded the number of females. If the null hypothesis of equal number of births is true, the probability of the observed outcome is 1/282, leading Arbuthnot to conclude that the probability of male and female births were not exactly equal. For his publications in 1692 and 1710, Arbuthnot is credited with "… the first use of significance tests …" , the first example of reasoning about statistical significance and moral certainty, and "… perhaps the first published report of a nonparametric test …". Hald further describes the impact of Arbuthnot's research. "Nicholas Bernoulli (1710–1713) completes the analysis of Arbuthnot's data by showing that the larger part of the variation of the yearly number of male births can be explained as binomial with ''p'' = 18/35. This is the first example of fitting a binomial to data. Hence we here have a test of significance rejecting the hypothesis ''p'' = 0.5 followed by an estimation of p and a discussion of the goodness of fit …"Relationship to other statistical tests
Wilcoxon signed-rank test
The sign test requires only that the observations in a pair be ordered, for example ''x'' > ''y''. In some cases, the observations for all subjects can be assigned a rank value (1, 2, 3, ...). If the observations can be ranked, and each observation in a pair is a random sample from a symmetric distribution, then the Wilcoxon signed-rank test is appropriate. The Wilcoxon test will generally have greater power to detect differences than the sign test. The asymptotic relative efficiency of the sign test to the Wilcoxon signed rank test, under these circumstances, is 0.67.Paired t-test
If the paired observations are numeric quantities (such as the actual length of the hind leg and foreleg in the Zar example), and the differences between paired observations are random samples from a single normal distribution, then the paired t-test is appropriate. The paired t-test will generally have greater power to detect differences than the sign test. The asymptotic relative efficiency of the sign test to the paired t-test, under these circumstances, is 0.637. However, if the distribution of the differences between pairs is not normal, but instead is heavy-tailed ( platykurtic distribution), the sign test can have more power than the paired t-test, with asymptotic relative efficiency of 2.0 relative to the paired t-test and 1.3 relative to the Wilcoxon signed rank test.McNemar's test
In some applications, the observations within each pair can only take the values 0 or 1. For example, 0 may indicate failure and 1 may indicate success. There are 4 possible pairs: , , , and . In these cases, the same procedure as the sign test is used, but is known asTrinomial test
Bian, McAleer and Wong proposed in 2011 a non-parametric test for paired data when there are many ties. They showed that their trinomial test is superior to the sign test in presence of ties.See also
* Wilcoxon signed-rank test – A more powerful variant of the sign test, but one which also assumes a symmetric distribution and interval data. * Median test – An unpaired alternative to the sign test.References
* Gibbons, J.D. and Chakraborti, S. (1992). Nonparametric Statistical Inference. Marcel Dekker Inc., New York. * Kitchens, L.J.(2003). ''Basic Statistics and Data Analysis.'' Duxbury. * Conover, W. J. (1980). ''Practical Nonparametric Statistics'', 2nd ed. Wiley, New York. * Lehmann, E. L. (1975). ''Nonparametrics: Statistical Methods Based on Ranks.'' Holden and Day, San Francisco. {{statistics, inference, collapsed Statistical tests Nonparametric statistics Articles with example R code