Evaluation of Subject-Specific Heuristics for Initial Learning Environments: A Pilot Study

McKay, Fraser and Kölling, Michael (2012) Evaluation of Subject-Specific Heuristics for Initial Learning Environments: A Pilot Study. In: Psychology of Programming Interest Group 24th Annual Conference, 21st-23rd November 2012, London, UK. (Full text available)

PDF (colour) - Accepted Version
Download (694kB) Preview
[img]
Preview

Abstract

Heuristic evaluation is a “discount” technique for finding usability problems in well-established domains. This paper presents thirteen suggested heuristics for initial learning environments (ILEs). To investigate the usefulness of these heuristics to other developers, we conducted a pilot study that compared two groups of evaluators: one using an older, generalised set of heuristics from the literature, and one using our domain-specific heuristics. In this study, we compare not just the number of problems found, but the way in which the problem reports were expressed. There was a significant difference in the length of written comments when problems were found (those from the new set being longer). New-set reviews touch on more themes – many make suggestions about what would improve the problem; many comments refer to a suggested cause-and-effect relationship. As designers, we find this detail helpful in understanding problems. Quantitative data from this study is not large enough to support any robust conclusions about the relative thoroughness of the heuristics at this time, but we plan to use lessons learned from this study in a larger version shortly.

Item Type: Conference or workshop item (Paper)
Uncontrolled keywords: Heuristic evaluation, computing education, novice programming systems, psychology of programming
Subjects: Q Science > QA Mathematics (inc Computing science) > QA 76 Software, computer programming, > QA76.9.H85 Human computer interaction
Divisions: Faculties > Science Technology and Medical Studies > School of Computing > Computing Education Group
Depositing User: F. Mckay
Date Deposited: 12 Nov 2012 10:09
Last Modified: 22 Mar 2013 16:19
Resource URI: http://kar.kent.ac.uk/id/eprint/32143 (The current URI for this page, for reference purposes)
  • Depositors only (login required):

Downloads

Downloads per month over past year