Within
psychometrics
Psychometrics is a field of study within psychology concerned with the theory and technique of measurement. Psychometrics generally refers to specialized fields within psychology and education devoted to testing, measurement, assessment, and ...
, Item analysis refers to statistical methods used for selecting items for inclusion in a psychological test. The concept goes back at least to Guildford (1936). The process of item analysis varies depending on the psychometric model. For example,
classical test theory
Classical test theory (CTT) is a body of related psychometric theory that predicts outcomes of psychological testing such as the difficulty of items or the ability of test-takers. It is a theory of testing based on the idea that a person's observe ...
or the
Rasch model
The Rasch model, named after Georg Rasch, is a psychometric model for analyzing categorical data, such as answers to questions on a reading assessment or questionnaire responses, as a function of the trade-off between the respondent's abilities, at ...
call for different procedures. In all cases, however, the purpose of item analysis is to produce a relatively short list of items (that is, questions to be included in an interview or questionnaire) that constitute a pure but comprehensive test of one or a few psychological constructs.
To carry out the analysis, a large pool of candidate items, all of which show some degree of
face validity Face validity is the extent to which a test is subjectively viewed as covering the concept it purports to measure. It refers to the transparency or relevance of a test as it appears to test participants. In other words, a test can be said to have f ...
, are given to a large sample of participants who are representative of the target population. Ideally, there should be at least 10 times as many candidate items as the desired final length of the test, and several times more people in the sample than there are items in the pool. Researchers apply a variety of statistical procedures to the responses to eliminate unsatisfactory items. For example, under classical test theory, researcher discard items if the answers:
* Show little variation within the sample
* Are strongly correlated with one or more other items
* Weakly correlate with the totality of the remaining items, reflected in an increase in
Cronbach's alpha
Cronbach's alpha (Cronbach's \alpha), also known as tau-equivalent reliability (\rho_T) or coefficient alpha (coefficient \alpha), is a reliability coefficient that provides a method of measuring internal consistency of tests and measures. Numero ...
if the item is eliminated from the test
In practical test construction, item analysis is an iterative process, and cannot be entirely automated. The psychometrician's judgement is required to determine whether the emerging set of items to be retained constitutes a satisfactory test of the target construct.{{Citation needed, date=August 2016 The three criteria above do not always agree, and a balance must be struck between them in deciding whether or not to include an item.
References
*Guilford, J. P. (1936). Psychometric methods. New York: McGraw-Hill.
*Kline, P. (1986). A handbook of test construction. London: Methuen
Psychometrics