How IRT can solve problems of ipsative data in forced-choice questionnaires

Brown, Anna and Maydeu-Olivares, Alberto (2013) How IRT can solve problems of ipsative data in forced-choice questionnaires. Psychological Methods, 18 (1). pp. 36-52. ISSN 1082-989X. (doi:https://doi.org/10.1037/a0030641) (Full text available)

PDF
Download (372kB) Preview
[img]
Preview
Official URL
http://dx.doi.org/10.1037/a0030641

Abstract

In multidimensional forced-choice (MFC) questionnaires, items measuring different attributes are presented in blocks, and participants have to rank-order the items within each block (fully or partially). Such comparative formats can reduce the impact of numerous response biases often affecting single-stimulus items (aka, rating or Likert scales). However, if scored with traditional methodology, MFC instruments produce ipsative data, whereby all individuals have a common total test score. Ipsative scoring distorts individual profiles (it is impossible to achieve all high or all low scale scores), construct validity (covariances between scales must sum to zero), criterion related validity (validity coefficients must sum to zero), and reliability estimates. We argue that these problems are caused by inadequate scoring of forced-choice items, and advocate the use of item response theory (IRT) models based on an appropriate response process for comparative data, such as Thurstone’s Law of Comparative Judgment. We show that by applying Thurstonian IRT modeling (Brown & Maydeu-Olivares, 2011), even existing forced-choice questionnaires with challenging features can be scored adequately and that the IRT-estimated scores are free from the problems of ipsative data.

Item Type: Article
Uncontrolled keywords: forced-choice format, ipsative data, multidimensional IRT, Thurstonian IRT model, Thurstonian factor model
Subjects: B Philosophy. Psychology. Religion > BF Psychology
H Social Sciences > HA Statistics
Divisions: Faculties > Social Sciences > School of Psychology > Applied Psychology
Depositing User: Anna Brown
Date Deposited: 19 Dec 2012 11:38 UTC
Last Modified: 28 May 2013 13:58 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/30530 (The current URI for this page, for reference purposes)
  • Depositors only (login required):

Downloads

Downloads per month over past year