Skip to main content
Kent Academic Repository

Heuristic Evaluation for Novice Programming Systems

Kölling, Michael, McKay, Fraser (2016) Heuristic Evaluation for Novice Programming Systems. Transactions of Computing Education, 16 (3). ISSN 1946-6226. (doi:10.1145/2872521) (KAR id:55885)


The past few years has seen a proliferation of novice programming tools. The availability of a large number of systems has made it difficult for many users to choose among them. Even for education researchers, comparing the relative quality of these tools, or judging their respective suitability for a given context, is hard in many instances. For designers of such systems, assessing the respective quality of competing design decisions can be equally difficult.

Heuristic evaluation provides a practical method of assessing the quality of alternatives in these situations and of identifying potential problems with existing systems for a given target group or context. Existing sets of heuristics, however, are not specific to the domain of novice programming and thus do not evaluate all aspects of interest to us in this specialised application domain.

In this article, we propose a set of heuristics to be used in heuristic evaluations of novice programming systems. These heuristics have the potential to allow a useful assessment of the quality of a given system with lower cost than full formal user studies and greater precision than the use of existing sets of heuristics. The heuristics are described and discussed in detail. We present an evaluation of the effectiveness of the heuristics that suggests that the new set of heuristics provides additional useful information to designers not obtained with existing heuristics sets.

Item Type: Article
DOI/Identification number: 10.1145/2872521
Uncontrolled keywords: HCI, heuristic evaluation, introductory programming tools
Subjects: Q Science > QA Mathematics (inc Computing science) > QA 76 Software, computer programming, > QA76.76 Computer software
Q Science > QA Mathematics (inc Computing science) > QA 76 Software, computer programming, > QA76.9.H85 Human computer interaction
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Computing
Depositing User: Michael Kolling
Date Deposited: 10 Jun 2016 17:17 UTC
Last Modified: 16 Feb 2021 13:35 UTC
Resource URI: (The current URI for this page, for reference purposes)

University of Kent Author Information

Kölling, Michael.

Creator's ORCID:
CReDIT Contributor Roles:

McKay, Fraser.

Creator's ORCID:
CReDIT Contributor Roles:
  • Depositors only (login required):

Total unique views for this document in KAR since July 2020. For more details click on the image.