Skip to main content

Relational approach to knowledge engineering for POMDP-based assistance systems as a translation of a psychological model

Grzes, Marek, Hoey, Jesse, Khan, Shehroz S., Mihailidis, Alex, Czarnuch, Stephen, Jackson, Dan, Monk, Andrew (2014) Relational approach to knowledge engineering for POMDP-based assistance systems as a translation of a psychological model. International Journal of Approximate Reasoning, 55 (1). pp. 36-58. ISSN 0888-613X. (doi:10.1016/j.ijar.2013.03.006) (KAR id:47510)

PDF (The final version from Elsevier) Publisher pdf
Language: English
Download (2MB) Preview
[thumbnail of The final version from Elsevier]
This file may not be suitable for users of assistive technology.
Request an accessible format
Official URL


Assistive systems for persons with cognitive disabilities (e.g. dementia) are difficult to build due to the wide range of different approaches people can take to accomplishing the same task, and the significant uncertainties that arise from both the unpredictability of client's behaviours and from noise in sensor readings. Partially observable Markov decision process (POMDP) models have been used successfully as the reasoning engine behind such assistive systems for small multi-step tasks such as hand washing. POMDP models are a powerful, yet flexible framework for modelling assistance that can deal with uncertainty and utility. Unfortunately, POMDPs usually require a very labour intensive, manual procedure for their definition and construction. Our previous work has described a knowledge driven method for automatically generating POMDP activity recognition and context sensitive prompting systems for complex tasks. We call the resulting POMDP a SNAP (SyNdetic Assistance Process). The spreadsheet-like result of the analysis does not correspond to the POMDP model directly and the translation to a formal POMDP representation is required. To date, this translation had to be performed manually by a trained POMDP expert. In this paper, we formalise and automate this translation process using a probabilistic relational model (PRM) encoded in a relational database. The database encodes the relational skeleton of the PRM, and includes the goals, action preconditions, environment states, cognitive model, client and system actions (i.e., the outcome of the SNAP analysis), as well as relevant sensor models. The database is easy to approach for someone who is not an expert in POMDPs, allowing them to fill in the necessary details of a task using a simple and intuitive procedure. The database, when filled, implicitly defines a ground instance of the relational skeleton, which we extract using an automated procedure, thus generating a POMDP model of the assistance task. A strength of the database is that it allows constraints to be specified, such that we can verify the POMDP model is, indeed, valid for the task given the analysis. We demonstrate the method by eliciting three assistance tasks from non-experts: handwashing, and toothbrushing for elderly persons with dementia, and on a factory assembly task for persons with a cognitive disability. We validate the resulting POMDP models using case-based simulations to show that they are reasonable for the domains. We also show a complete case study of a designer specifying one database, including an evaluation in a real-life experiment with a human actor.

Item Type: Article
DOI/Identification number: 10.1016/j.ijar.2013.03.006
Uncontrolled keywords: dynamic Bayesian networks, knowledge engineering, probabilistic planning, POMDPs, reinforcement learning, COACH, assistance systems, handwashing, prompting system, SQL, relational database
Subjects: Q Science > Q Science (General) > Q335 Artificial intelligence
Q Science > QA Mathematics (inc Computing science) > QA273 Probabilities
Q Science > QA Mathematics (inc Computing science) > QA276 Mathematical statistics
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Computing
Depositing User: Marek Grzes
Date Deposited: 02 Mar 2015 21:27 UTC
Last Modified: 16 Feb 2021 13:23 UTC
Resource URI: (The current URI for this page, for reference purposes)
Grzes, Marek:
  • Depositors only (login required):


Downloads per month over past year