HOME

TheInfoList



OR:

Pairwise comparison generally is any process of comparing entities in pairs to judge which of each entity is preferred, or has a greater amount of some
quantitative property Quantitative research is a research strategy that focuses on quantifying the collection and analysis of data. It is formed from a deductive approach where emphasis is placed on the testing of theory, shaped by empiricist and positivist philos ...
, or whether or not the two entities are identical. The method of pairwise comparison is used in the scientific study of
preference In psychology, economics and philosophy, preference is a technical term usually used in relation to choosing between alternatives. For example, someone prefers A over B if they would rather choose A than B. Preferences are central to decision the ...
s, attitudes,
voting systems An electoral or voting system is a set of rules used to determine the results of an election. Electoral systems are used in politics to elect governments, while non-political elections may take place in business, nonprofit organizations and inf ...
,
social choice Social choice theory is a branch of welfare economics that extends the theory of rational choice to collective decision-making. Social choice studies the behavior of different mathematical procedures ( social welfare functions) used to combine i ...
,
public choice Public choice, or public choice theory, is "the use of economic tools to deal with traditional problems of political science."Gordon Tullock, 9872008, "public choice," ''The New Palgrave Dictionary of Economics''. . It includes the study of po ...
,
requirements engineering Requirements engineering (RE) is the process of defining, documenting, and maintaining requirements in the engineering design process. It is a common role in systems engineering and software engineering. The first use of the term ''requiremen ...
and multiagent AI systems. In
psychology Psychology is the scientific study of mind and behavior. Its subject matter includes the behavior of humans and nonhumans, both consciousness, conscious and Unconscious mind, unconscious phenomena, and mental processes such as thoughts, feel ...
literature, it is often referred to as paired comparison. Prominent psychometrician L. L. Thurstone first introduced a scientific approach to using pairwise comparisons for measurement in 1927, which he referred to as the
law of comparative judgment The law of comparative judgment was conceived by L. L. Thurstone. In modern-day terminology, it is more aptly described as a model that is used to obtain measurements from any process of pairwise comparison. Examples of such processes are the compa ...
. Thurstone linked this approach to psychophysical theory developed by
Ernst Heinrich Weber Ernst Heinrich Weber (; ; 24 June 1795 – 26 January 1878) was a German physician who is considered one of the founders of experimental psychology. Ernst Weber was born into an academic background, with his father serving as a professor at t ...
and
Gustav Fechner Gustav Theodor Fechner (; ; 19 April 1801 – 18 November 1887) was a German physicist, philosopher, and experimental psychologist. A pioneer in experimental psychology and founder of psychophysics (techniques for measuring the mind), he inspi ...
. Thurstone demonstrated that the method can be used to order items along a dimension such as preference or importance using an interval-type scale. Mathematician
Ernst Zermelo Ernst Friedrich Ferdinand Zermelo (; ; 27 July 187121 May 1953) was a German logician and mathematician, whose work has major implications for the foundations of mathematics. He is known for his role in developing Zermelo–Fraenkel set theory, Z ...
(1929) first described a model for pairwise comparisons for
chess ranking Chess is a board game for two players. It is an abstract strategy game that involves Perfect information, no hidden information and no elements of game of chance, chance. It is played on a square chessboard, board consisting of 64 squares arran ...
in incomplete tournaments, which serves as the basis (even though not credited for a while) for methods such as the
Elo rating system The Elo rating system is a method for calculating the relative skill levels of players in zero-sum games such as chess or esports. It is named after its creator Arpad Elo, a Hungarian-American chess master and physics professor. The Elo system wa ...
and is equivalent to the
Bradley–Terry model The Bradley–Terry model is a probability model for the outcome of pairwise comparisons between items, teams, or objects. Given a pair of items and drawn from some population, it estimates the probability that the pairwise comparison turns out ...
that was proposed in 1952.


Overview

If an individual or organization expresses a preference between two mutually distinct alternatives, this preference can be expressed as a pairwise comparison. If the two alternatives are ''x'' and ''y'', the following are the possible pairwise comparisons: The agent prefers ''x'' over ''y'': "''x'' > ''y''" or "''xPy''" The agent prefers ''y'' over ''x'': "''y'' > ''x''" or "''yPx''" The agent is indifferent between both alternatives: "''x'' = ''y''" or "''xIy''"


Probabilistic models

In terms of modern psychometric theory probabilistic models, which include Thurstone's approach (also called the law of comparative judgment), the Bradley–Terry–Luce (BTL) model, and general
stochastic transitivity Stochastic transitivity models are stochastic versions of the transitivity property of binary relations studied in mathematics. Several models of stochastic transitivity exist and have been used to describe the probabilities involved in experiments ...
models, are more aptly regarded as measurement models. The Bradley–Terry–Luce (BTL) model is often applied to pairwise comparison data to scale preferences. The BTL model is identical to Thurstone's model if the simple
logistic function A logistic function or logistic curve is a common S-shaped curve ( sigmoid curve) with the equation f(x) = \frac where The logistic function has domain the real numbers, the limit as x \to -\infty is 0, and the limit as x \to +\infty is L. ...
is used. Thurstone used the normal distribution in applications of the model. The simple logistic function varies by less than 0.01 from the cumulative normal
ogive An ogive ( ) is the roundly tapered end of a two- or three-dimensional object. Ogive curves and surfaces are used in engineering, architecture, woodworking, and ballistics. Etymology The French Orientalist Georges Séraphin Colin gives as ...
across the range, given an arbitrary scale factor. In the BTL model, the probability that object ''j'' is judged to have more of an attribute than object ''i'' is: : \Pr \ =\frac = \sigma (\delta_j - \delta_i), where \delta_i is the scale location of object ''i''; \sigma is the
logistic function A logistic function or logistic curve is a common S-shaped curve ( sigmoid curve) with the equation f(x) = \frac where The logistic function has domain the real numbers, the limit as x \to -\infty is 0, and the limit as x \to +\infty is L. ...
(the inverse of the
logit In statistics, the logit ( ) function is the quantile function associated with the standard logistic distribution. It has many uses in data analysis and machine learning, especially in Data transformation (statistics), data transformations. Ma ...
). For example, the scale location might represent the perceived quality of a product, or the perceived weight of an object. The BTL model, the Thurstonian model as well as the
Rasch model The Rasch model, named after Georg Rasch, is a psychometric model for analyzing categorical data, such as answers to questions on a reading assessment or questionnaire responses, as a function of the trade-off between the respondent's abilities, ...
for measurement are all closely related and belong to the same class of
stochastic transitivity Stochastic transitivity models are stochastic versions of the transitivity property of binary relations studied in mathematics. Several models of stochastic transitivity exist and have been used to describe the probabilities involved in experiments ...
. Thurstone used the method of pairwise comparisons as an approach to measuring perceived intensity of physical stimuli, attitudes, preferences, choices, and values. He also studied implications of the theory he developed for opinion polls and political voting (Thurstone, 1959).


Transitivity

For a given decision agent, if the information, objective, and alternatives used by the agent remain constant, then it is generally assumed that pairwise comparisons over those alternatives by the decision agent are transitive. Most agree upon what transitivity is, though there is debate about the transitivity of indifference. The rules of transitivity are as follows for a given decision agent. * If xPy and yPz, then xPz * If xPy and yIz, then xPz * If xIy and yPz, then xPz * If xIy and yIz, then xIz This corresponds to (xPy or xIy) being a total preorder, P being the corresponding
strict weak order In mathematics, especially order theory, a weak ordering is a mathematical formalization of the intuitive notion of a ranking of a set, some of whose members may be tied with each other. Weak orders are a generalization of totally ordered set ...
, and I being the corresponding
equivalence relation In mathematics, an equivalence relation is a binary relation that is reflexive, symmetric, and transitive. The equipollence relation between line segments in geometry is a common example of an equivalence relation. A simpler example is equ ...
. Probabilistic models also give rise to stochastic variants of transitivity, all of which can be verified to satisfy (non-stochastic) transitivity within the bounds of errors of estimates of scale locations of entities. Thus, decisions need not be deterministically transitive in order to apply probabilistic models. However, transitivity will generally hold for a large number of comparisons if models such as the BTL can be effectively applied. Using a transitivity testNikolić D (2012) Non-parametric detection of temporal order across pairwise measurements of time delays. ''Journal of Computational Neuroscience'', 22(1)" pp. 5–19. http://www.danko-nikolic.com/wp-content/uploads/2011/09/Nikolic-Transitivity-2007.pdf one can investigate whether a data set of pairwise comparisons contains a higher degree of transitivity than expected by chance.


Argument for intransitivity of indifference

Some contend that indifference is not transitive. Consider the following example. Suppose you like apples and you prefer apples that are larger. Now suppose there exists an apple A, an apple B, and an apple C which have identical intrinsic characteristics except for the following. Suppose B is larger than A, but it is not discernible without an extremely sensitive scale. Further suppose C is larger than B, but this also is not discernible without an extremely sensitive scale. However, the difference in sizes between apples A and C is large enough that you can discern that C is larger than A without a sensitive scale. In psychophysical terms, the size difference between A and C is above the just noticeable difference ('jnd') while the size differences between A and B and B and C are below the jnd. You are confronted with the three apples in pairs without the benefit of a sensitive scale. Therefore, when presented A and B alone, you are indifferent between apple A and apple B; and you are indifferent between apple B and apple C when presented B and C alone. However, when the pair A and C are shown, you prefer C over A.


Preference orders

If pairwise comparisons are in fact transitive in respect to the four mentioned rules, then pairwise comparisons for a list of alternatives (''A''1, ''A''2, ''A''3, ..., ''A''''n''−1, and ''A''''n'') can take the form: : ''A''1(>
XOR Exclusive or, exclusive disjunction, exclusive alternation, logical non-equivalence, or logical inequality is a logical operator whose negation is the logical biconditional. With two inputs, XOR is true if and only if the inputs differ (one ...
=)''A''2(>
XOR Exclusive or, exclusive disjunction, exclusive alternation, logical non-equivalence, or logical inequality is a logical operator whose negation is the logical biconditional. With two inputs, XOR is true if and only if the inputs differ (one ...
=)''A''3(>
XOR Exclusive or, exclusive disjunction, exclusive alternation, logical non-equivalence, or logical inequality is a logical operator whose negation is the logical biconditional. With two inputs, XOR is true if and only if the inputs differ (one ...
=) ... (>
XOR Exclusive or, exclusive disjunction, exclusive alternation, logical non-equivalence, or logical inequality is a logical operator whose negation is the logical biconditional. With two inputs, XOR is true if and only if the inputs differ (one ...
=)''A''''n''−1(>
XOR Exclusive or, exclusive disjunction, exclusive alternation, logical non-equivalence, or logical inequality is a logical operator whose negation is the logical biconditional. With two inputs, XOR is true if and only if the inputs differ (one ...
=)''A''''n'' For example, if there are three alternatives ''a'', ''b'', and ''c'', then the possible preference orders are: *a>b>c *a>c>b *b>a>c *b>c>a *c>a>b *c>b>a *a>b=c *b=c>a *b>a=c *a=c>b *c>a=b *a=b>c *a=b=c If the number of alternatives is n, and indifference is not allowed, then the number of possible preference orders for any given ''n''-value is ''n''!. If indifference is allowed, then the number of possible preference orders is the number of total preorders. It can be expressed as a function of n: : \sum_^n k! S_2(n,k), where ''S''2(''n'', ''k'') is the Stirling number of the second kind.


Applications

One important application of pairwise comparisons is the widely used
Analytic Hierarchy Process In the theory of decision making, the analytic hierarchy process (AHP), also analytical hierarchy process, is a structured technique for organizing and analyzing MCDA, complex decisions, based on mathematics and psychology. It was developed by ...
, a structured technique for helping people deal with complex decisions. It uses pairwise comparisons of tangible and intangible factors to construct ratio scales that are useful in making important decisions. Another important application is the Potentially All Pairwise RanKings of all possible Alternatives (PAPRIKA) method. The method involves the decision-maker repeatedly pairwise comparing and ranking alternatives defined on two criteria or attributes at a time and involving a trade-off, and then, if the decision-maker chooses to continue, pairwise comparisons of alternatives defined on successively more criteria. From the pairwise rankings, the relative importance of the criteria to the decision-maker, represented as weights, is determined.


See also

*
Analytic Hierarchy Process In the theory of decision making, the analytic hierarchy process (AHP), also analytical hierarchy process, is a structured technique for organizing and analyzing MCDA, complex decisions, based on mathematics and psychology. It was developed by ...
(AHP) *
Law of comparative judgment The law of comparative judgment was conceived by L. L. Thurstone. In modern-day terminology, it is more aptly described as a model that is used to obtain measurements from any process of pairwise comparison. Examples of such processes are the compa ...
* Potentially all pairwise rankings of all possible alternatives (PAPRIKA) method * PROMETHEE pairwise comparison method *
Preference (economics) In economics, and in other social sciences, preference refers to an order by which an Agent (economics), agent, while in search of an "optimal choice", ranks alternatives based on their respective utility. ''Preferences'' are evaluations that conc ...
*
Stochastic Transitivity Stochastic transitivity models are stochastic versions of the transitivity property of binary relations studied in mathematics. Several models of stochastic transitivity exist and have been used to describe the probabilities involved in experiments ...
*
Condorcet method A Condorcet method (; ) is an election method that elects the candidate who wins a majority of the vote in every head-to-head election against each of the other candidates, whenever there is such a candidate. A candidate with this property, the ...


References

* * {{Cite OEIS, sequencenumber=A000670, name=Number of preferential arrangements of n labeled elements * Y. Chevaleyre, P.E. Dunne, U. Endriss, J. Lang, M. Lemaître, N. Maudet, J. Padget, S. Phelps, J.A. Rodríguez-Aguilar, and P. Sousa. Issues in Multiagent Resource Allocation. Informatica, 30:3–31, 2006.


Further reading

*Bradley, R.A. and Terry, M.E. (1952). Rank analysis of incomplete block designs, I. the method of paired comparisons. ''Biometrika'', 39, 324–345. *David, H.A. (1988). The Method of Paired Comparisons. New York: Oxford University Press. *Luce, R.D. (1959). ''Individual Choice Behaviours'': A Theoretical Analysis. New York: J. Wiley. *Thurstone, L.L. (1927). A law of comparative judgement. ''Psychological Review'', 34, 278–286. *Thurstone, L.L. (1929). ''The Measurement of Psychological Value''. In T.V. Smith and W.K. Wright (Eds.), Essays in Philosophy by Seventeen Doctors of Philosophy of the University of Chicago. Chicago: Open Court. *Thurstone, L.L. (1959). ''The Measurement of Values''. Chicago: The University of Chicago Press. *Zermelo, E. (1928).
Die Berechnung der Turnier-Ergebnisse als ein Maximumproblem der Wahrscheinlichkeitsrechnung
', Mathematische Zeitschrift 29, 1929, S. 436–460 Psychometrics