Skip to main content

Heuristic Evaluation for Novice Programming Systems

Kölling, Michael, McKay, Fraser (2016) Heuristic Evaluation for Novice Programming Systems. Transactions of Computing Education, 16 (3). ISSN 1946-6226. (doi:10.1145/2872521)

PDF - Author's Accepted Manuscript
Download (987kB) Preview
Official URL


The past few years has seen a proliferation of novice programming tools. The availability of a large number of systems has made it difficult for many users to choose among them. Even for education researchers, comparing the relative quality of these tools, or judging their respective suitability for a given context, is hard in many instances. For designers of such systems, assessing the respective quality of competing design decisions can be equally difficult. Heuristic evaluation provides a practical method of assessing the quality of alternatives in these situations and of identifying potential problems with existing systems for a given target group or context. Existing sets of heuristics, however, are not specific to the domain of novice programming and thus do not evaluate all aspects of interest to us in this specialised application domain. In this article, we propose a set of heuristics to be used in heuristic evaluations of novice programming systems. These heuristics have the potential to allow a useful assessment of the quality of a given system with lower cost than full formal user studies and greater precision than the use of existing sets of heuristics. The heuristics are described and discussed in detail. We present an evaluation of the effectiveness of the heuristics that suggests that the new set of heuristics provides additional useful information to designers not obtained with existing heuristics sets.

Item Type: Article
DOI/Identification number: 10.1145/2872521
Uncontrolled keywords: HCI, heuristic evaluation, introductory programming tools
Subjects: Q Science > QA Mathematics (inc Computing science) > QA 76 Software, computer programming, > QA76.76 Computer software
Q Science > QA Mathematics (inc Computing science) > QA 76 Software, computer programming, > QA76.9.H85 Human computer interaction
Divisions: Faculties > Sciences > School of Computing > Computing Education Group
Depositing User: Michael Kolling
Date Deposited: 10 Jun 2016 17:17 UTC
Last Modified: 29 May 2019 17:28 UTC
Resource URI: (The current URI for this page, for reference purposes)
  • Depositors only (login required):


Downloads per month over past year