Skip to main content

Evaluation of Subject-Specific Heuristics for Initial Learning Environments: A Pilot Study

McKay, Fraser, Kölling, Michael (2012) Evaluation of Subject-Specific Heuristics for Initial Learning Environments: A Pilot Study. In: Proceedings of the 24th Psychology of Programming Interest Group Annual Conference 2012. . pp. 128-138. London Metropolitan University, London, UK (KAR id:32143)

PDF (colour) Author's Accepted Manuscript
Language: English
Download (375kB) Preview
[thumbnail of colour]
This file may not be suitable for users of assistive technology.
Request an accessible format


Heuristic evaluation is a “discount” technique for finding usability problems in well-established domains. This paper presents thirteen suggested heuristics for initial learning environments (ILEs). To investigate the usefulness of these heuristics to other developers, we conducted a pilot study that compared two groups of evaluators: one using an older, generalised set of heuristics from the literature, and one using our domain-specific heuristics. In this study, we compare not just the number of problems found, but the way in which the problem reports were expressed. There was a significant difference in the length of written comments when problems were found (those from the new set being longer). New-set reviews touch on more themes – many make suggestions about what would improve the problem; many comments refer to a suggested cause-and-effect relationship. As designers, we find this detail helpful in understanding problems. Quantitative data from this study is not large enough to support any robust conclusions about the relative thoroughness of the heuristics at this time, but we plan to use lessons learned from this study in a larger version shortly.

Item Type: Conference or workshop item (Paper)
Uncontrolled keywords: Heuristic evaluation, computing education, novice programming systems, psychology of programming
Subjects: Q Science > QA Mathematics (inc Computing science) > QA 76 Software, computer programming, > QA76.9.H85 Human computer interaction
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Computing
Depositing User: F. Mckay
Date Deposited: 12 Nov 2012 10:09 UTC
Last Modified: 16 Nov 2021 10:09 UTC
Resource URI: (The current URI for this page, for reference purposes)
  • Depositors only (login required):