An instructional
manipulation check
Manipulation check is a term in experimental research in the social sciences which refers to certain kinds of secondary evaluations of an experiment.
Overview
Manipulation checks are measured variables that show what the manipulated variables co ...
, often abbreviated IMC, is a special kind of question inserted in a
questionnaire among the regular questions, designed to check whether respondents are paying attention to the instructions.
Discarding responses by participants who fail to read the instructions reduces the
signal-to-noise ratio and can thereby increase the
statistical power
In statistics, the power of a binary hypothesis test is the probability that the test correctly rejects the null hypothesis (H_0) when a specific alternative hypothesis (H_1) is true. It is commonly denoted by 1-\beta, and represents the chances ...
of an experiment. The tool was developed by Oppenheimer ''et al.''
Eliminating random responses this way before performing
statistical hypothesis testing
A statistical hypothesis test is a method of statistical inference used to decide whether the data at hand sufficiently support a particular hypothesis.
Hypothesis testing allows us to make probabilistic statements about population parameters.
...
may be considered a legitimate form of
data manipulation, but should be duly mentioned in publications reporting on the outcome of the experiment in question.
Blue-dot task
Among several forms an IMC can take, a popular one is the so-called blue-dot task,
[ suitable for on-line questionnaires. A number of larger blue circles are arranged according to a ]Likert scale
A Likert scale ( , commonly mispronounced as ) is a psychometric scale commonly involved in research that employs questionnaires. It is the most widely used approach to scaling responses in survey research, such that the term (or more fully the ...
from (say) "very rarely" to "very frequently". Participants who ignore the instructions and merely want to finish the task will just click any one of these circles. The instructions, however, ask the participants to ignore the larger circles and instead click a little blue dot at the bottom of the screen. Oppenheimer ''et al.'' report that in a large sample of undergraduate participants, approximately 7% failed this task.[
]
References
Questionnaire construction
Survey methodology
{{sci-stub