Comparing internet and face-to-face surveys as methods for eliciting preferences for social care-related quality of life: Evidence from England using the ASCOT service user measure

Purpose: Traditionally, researchers have relied on eliciting preferences through face-to-face interviews. Recently, there has been a shift towards using internet-based methods. Different methods of data collection may be a source of variation in the results. In this study, we compare the preferences for the Adult Social Care Outcomes Toolkit (ASCOT) service user measure elicited using best-worst scaling (BWS) via a face-to-face interview and an online survey.

Methods: Data were collected from a representative sample of the general population in England. The respondents (face-to-face: n=500; online: n=1,001) completed a survey, which included the BWS experiment involving the ASCOT measure. Each respondent received eight best-worst scenarios and made four choices (best, second best, worst, second worst) in each scenario. Multinomial logit regressions were undertaken to analyse the data taking into account differences in the characteristics of the two samples and the repeated nature of the data.

Results: We initially found a number of small significant differences in preferences between the two methods across all ASCOT domains. These differences were substantially reduced – from 15 to five out of 30 coefficients being different at the 5% level – and remained small in value after controlling for differences in observable and unobservable characteristics of the two samples.

Conclusions: This comparison demonstrates that face-to-face and internet surveys may lead to fairly similar preferences for social care-related quality of life when differences in sample characteristics are controlled for. With or without a constant sampling frame, studies should carefully design the BWS exercise and provide similar levels of clarification to participants in each survey to minimise the amount of error variance in the choice process.