Skip to main content
Kent Academic Repository

How IRT can solve problems of ipsative data in forced-choice questionnaires

Brown, Anna, Maydeu-Olivares, Alberto (2013) How IRT can solve problems of ipsative data in forced-choice questionnaires. Psychological Methods, 18 (1). pp. 36-52. ISSN 1082-989X. (doi:10.1037/a0030641) (KAR id:30530)


In multidimensional forced-choice (MFC) questionnaires, items measuring different attributes are presented in blocks, and participants have to rank-order the items within each block (fully or partially). Such comparative formats can reduce the impact of numerous response biases often affecting single-stimulus items (aka, rating or Likert scales). However, if scored with traditional methodology, MFC instruments produce ipsative data, whereby all individuals have a common total test score. Ipsative scoring distorts individual profiles (it is impossible to achieve all high or all low scale scores), construct validity (covariances between scales must sum to zero), criterion related validity (validity coefficients must sum to zero), and reliability estimates.

We argue that these problems are caused by inadequate scoring of forced-choice items, and advocate the use of item response theory (IRT) models based on an appropriate response process for comparative data, such as Thurstone’s Law of Comparative Judgment. We show that by applying Thurstonian IRT modeling (Brown & Maydeu-Olivares, 2011), even existing forced-choice questionnaires with challenging features can be scored adequately and that the IRT-estimated scores are free from the problems of ipsative data.

Item Type: Article
DOI/Identification number: 10.1037/a0030641
Uncontrolled keywords: forced-choice format, ipsative data, multidimensional IRT, Thurstonian IRT model, Thurstonian factor model
Subjects: B Philosophy. Psychology. Religion > BF Psychology
H Social Sciences > HA Statistics
Divisions: Divisions > Division of Human and Social Sciences > School of Psychology
Depositing User: Anna Brown
Date Deposited: 19 Dec 2012 11:38 UTC
Last Modified: 16 Nov 2021 10:08 UTC
Resource URI: (The current URI for this page, for reference purposes)

University of Kent Author Information

  • Depositors only (login required):

Total unique views for this document in KAR since July 2020. For more details click on the image.