Skip to main content
Kent Academic Repository

Multidimensional Forced-Choice CAT with Dominance Items: An Empirical Comparison with Optimal Static Testing under Different Desirability Matching

Lin, Yin, Brown, Anna, Williams, Paul (2022) Multidimensional Forced-Choice CAT with Dominance Items: An Empirical Comparison with Optimal Static Testing under Different Desirability Matching. Educational and Psychological Measurement, 83 (2). pp. 322-350. ISSN 0013-1644. E-ISSN 1552-3888. (doi:10.1177/00131644221077637) (KAR id:93137)

Abstract

Several forced-choice (FC) computerized adaptive tests (CATs) have emerged in the field of organizational psychology, all of them employing ideal-point items. However, despite most items developed historically follow dominance response models, research on FC CAT using dominance items is limited. Existing research is heavily dominated by simulations and lacking in empirical deployment. This empirical study trialed a FC CAT with dominance items described by the Thurstonian Item Response Theory model (Brown & Maydeu-Olivares, 2011) with research participants. This study investigated important practical issues such as the implications of adaptive item selection and social desirability balancing criteria on score distributions, measurement accuracy and participant perceptions. Moreover, non-adaptive but optimal tests of similar design were trialed alongside the CATs to provide a baseline for comparison, helping to quantify the return on investment when converting an otherwise-optimized static assessment into an adaptive one. While the benefit of adaptive item selection in improving measurement precision was confirmed, results also indicated that at shorter test lengths CAT had no notable advantage compared to optimal static tests. Taking a holistic view incorporating both psychometric and operational considerations, implications for the design and deployment of FC assessments in research and practice are discussed.

Item Type: Article
DOI/Identification number: 10.1177/00131644221077637
Additional information: For the purpose of open access, the author has applied a CC BY public copyright licence (where permitted by UKRI, an Open Government Licence or CC BY ND public copyright licence may be used instead) to any Author Accepted Manuscript version arising
Uncontrolled keywords: Forced choice, computerized adaptive testing, multidimensional item response theory, Thurstonian IRT model, personality.
Subjects: B Philosophy. Psychology. Religion > BF Psychology
Divisions: Divisions > Division of Human and Social Sciences > School of Psychology
Funders: [37325] UNSPECIFIED
Depositing User: Anna Brown
Date Deposited: 10 Feb 2022 13:23 UTC
Last Modified: 04 Mar 2024 16:17 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/93137 (The current URI for this page, for reference purposes)

University of Kent Author Information

Lin, Yin.

Creator's ORCID:
CReDIT Contributor Roles:

Brown, Anna.

Creator's ORCID: https://orcid.org/0000-0002-5612-2627
CReDIT Contributor Roles:
  • Depositors only (login required):

Total unique views for this document in KAR since July 2020. For more details click on the image.