Skip to main content
Kent Academic Repository

Investigation of Mobile Games for Cognitive Assessment and Screening with a Focus on Touch-based and Motion Features

Intarasirisawat, Jittrapol (2021) Investigation of Mobile Games for Cognitive Assessment and Screening with a Focus on Touch-based and Motion Features. Doctor of Philosophy (PhD) thesis, University of Kent,. (doi:10.22024/UniKent/01.02.92770) (KAR id:92770)


Early detection of cognitive decline is important for timely intervention and treatment strategies to prevent further deterioration or development of more severe forms of cognitive dysfunction. Therefore, many tests have been developed for screening and monitoring changes in cognitive status. However, these existing assessment and screening tools are not designed for self-administration without a trained examiner. Moreover, the lack of multiple variations of these paper-based measures and repeated exposure to such tests could reduce their sensitivity to detect cognitive changes due to practice effects. These limitations pose clinical challenges to early identification of cognitive deficits and monitoring of longitudinal changes in cognitive function, especially in resource-limited settings. To this end, a number of studies have adopted mobile technology and gamification to facilitate remote and self-administered cognitive assessment and screening in a less effortful and engaging manner. Despite this, existing literature has so far only examined the feasibility of using gameplay performance as a means for cognitive assessment. There has not been any attempt to explore gameplay behaviours as revealed through patterns of touch interactions and device motions as indicative features for cognitive evaluation. Therefore the aim of this thesis is to investigate the use of touch and motions features in game-based cognitive assessment and screening. This is achieved through two studies.

The first study was carried out to examine the links between cognitive abilities and underlying patterns of user-game interaction with a focus on touch gestures and device motions. Twenty-two healthy participants took part in the two-session experiment where they were asked to take a series of standard cognitive assessments followed by playing three casual mobile games in which user-game interaction data were passively collected. The results from bivariate analysis indicated that increases in swipe length and swipe speed, in the game context, were significantly correlated with declines in response inhibition ability but increased performance on attention. However, it remained unclear whether the device motion features alone could be used to identify cognitive ability as the results provide only weak evidence for relationships between cognitive performance and the underlying device motion patterns while playing the games.

In the second study, we evaluated the potential use of these behavioural features and mobile games as a potential screening tool for clinical conditions with cognitive impairment. Alcohol-related brain damage (ARBD) is often found to be associated with deficits in multiple cognitive functions in patients with alcohol dependence, which is the focus of this thesis. Based on findings from the preliminary study, the second experimental study was carried out to investigate the feasibility of using such user-game interaction patterns on mobile games to develop an automated screening tool for alcohol-dependent patients. The classification performance of various supervised learning algorithms was evaluated on data collected from 40 patients and 40 age-matched healthy adults. The results showed that patients with alcohol dependence could be automatically identified accurately using the ensemble of touch, device motion, and gameplay performance features on 3-minute samples (accuracy=0.95, sensitivity=0.95, and specificity=0.95).

The findings provide evidence suggesting the potential use of user-game interaction metrics on existing mobile games as discriminant features for developing an implicit measure to identify alcohol dependence conditions. In addition to supporting healthcare professionals in clinical decision-making, the game-based method could be used as a novel strategy to promote self-screening, especially outside of clinical settings. The findings from this thesis were also applied to guidelines to aid researchers in the game interaction design to capitalise on the use of touch and device motion features with regard to cognitive assessment and screening.

Item Type: Thesis (Doctor of Philosophy (PhD))
Thesis advisor: Ang, Chee Siang
DOI/Identification number: 10.22024/UniKent/01.02.92770
Uncontrolled keywords: Human-computer interaction (HCI), Touch Interaction, Machine Learning, Health Informatics, Cognitive assessment, Games, Alcohol screening
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Engineering and Digital Arts
SWORD Depositor: System Moodle
Depositing User: System Moodle
Date Deposited: 19 Jan 2022 15:10 UTC
Last Modified: 20 Jan 2022 10:03 UTC
Resource URI: (The current URI for this page, for reference purposes)

University of Kent Author Information

Intarasirisawat, Jittrapol.

Creator's ORCID:
CReDIT Contributor Roles:
  • Depositors only (login required):

Total unique views for this document in KAR since July 2020. For more details click on the image.