Measuring personality in the presence of faking

Every year millions of applicants to jobs or educational programs complete self-report measures of personality. Despite assumed validity and utility in selection, such measures are open to intentional manipulation by applicants who want to create a desired impression. These impression management behaviours are prevalent and typically involve exaggerating positive characteristics and downplaying negative ones in line with social norms (socially desirable responding) or perceived job/program demands (faking). Faking is motivated and purposeful behaviour that can result in substantial distortions to test scores, when they no longer reflect the attributes that the test intends to measure. The negative consequences of faking are well researched and documented, with the most fundamental problem being that faking changes the true ordering of applicants, thus destroying the validity and fairness of selection decisions.

This talk will provide an overview of research on measuring personality in high stakes, discussing the evolution of views on the impact of faking on test scores. I will present the emerging methods of detection and statistical control for faking, which incorporate faking behaviour in psychometric response models (Böckenholt, 2014; Brown & Böckenholt, 2022). These methods are revolutionary in overcoming the simplifying assumption that faking behaviour is consistent throughout the assessment, and offer a great promise as well as natural limitations in their applicability and scope. Unlike cheating on knowledge or ability test, faking behaviour on personality questionnaires cannot be observed and therefore is extremely challenging to identify reliably after it had occurred. This is why many practitioners are turning to methods that prevent faking from occurring in the first place.

In terms of faking prevention methods, I will focus on forced-choice questionnaires that make it impossible for a test taker to endorse all desirable characteristics. Popularity of such questionnaires have been growing steadily since proper scaling methods became available for them, overcoming the problems of ipsative (or relative-to-self, interpersonally incomparable) test scores (e.g. Stark, Chernyshenko, & Drasgow, 2005; Brown & Maydeu-Olivares, 2011). This facilitated rapid progress in development of new personality assessments that are more fake-resistant than the traditional Likert scales (e.g. Cao & Drasgow, 2019). However, many controversies and unanswered questions remain about the effectiveness of comparative judgements (of which forced choice is only one type) in preventing faking. Individual differences in cognitive ability, emotional intelligence, and ability to identify selection criteria, as well as item properties such as valence or ambiguity – all play part. I will report latest findings in this area and share my views and recommendations on the use of comparative judgements in personality assessments going forward.