Skip to main content

Logging student answer data in CALL exercises to gauge feedback efficacy

Fowler, Alison M.L. (2006) Logging student answer data in CALL exercises to gauge feedback efficacy. In: Colpaert, Jozef and Decoo, Wilfried and van Beuren, Saskia and Godfroid, Aline, eds. CALL & Monitoring the Learner - 12th International CALL Conference. Universiteit Antwerpen, Antwerpen, pp. 83-91. ISBN 90-5728-058-2. (KAR id:14446)

Language: English
Click to download this file (86kB) Preview
This file may not be suitable for users of assistive technology.
Request an accessible format


An important advantage of online assessment work is that answer data can be easily stored and later analysed with a view to establishing the efficacy of the assessment methodology. A 5-year study of the effectiveness of online grammar exercises has been carried out at the University of Kent. The exercises featured require input in the form of whole sentences (since this is a more authentic test of language skills than single word input or multiple choice). Error feedback is generic (indicating where errors have occurred) rather than specific (indicating the exact nature of the errors) because the error-diagnosis system has been designed to be completely language independent. The study aimed to gauge whether this type of feedback is effective - in terms of enabling students to:

· identify the types of mistakes in their input;

· rectify the mistakes;

· learn from the mistakes and apply that learning to subsequent problems

There was initial concern that the generic feedback might not provide enough detail to enable users to understand and correct their errors - however extensive use by the University's Spanish department has shown that this type of mark-up is very effective. Chapelle (1998) stresses that it is important for learners to be given the opportunity to correct their linguistic errors. Users of this system, having failed to answer a question correctly on their first attempt, are permitted a second attempt. It is abundantly clear from the logged data that where users make mistakes in their first attempt (and they generally do since the material is designed to be testing), there is almost always a significant improvement in attempt two. This alone would be enough to show that the feedback mode is effective. However this was not enough to prove the pedagogical efficacy of this means of exercise presentation. Therefore more detailed analysis was performed. Over several years of trials more than 100,000 answers have been logged and every answer has been analysed. It can be shown that, for well-designed exercises, as students progress through an exercise they improve in three ways:

· more questions are answered correctly on the first attempt;

· overall questions scores (i.e. the average of 1<sup>s</sup>t and 2<sup>n</sup>d attempts at questions) improve;

· thinking time for formulating answers decreases

The degree of increase in accuracy and decrease in thinking time is exercise-dependent, but the overall picture shows clearly that the generic, language-independent feedback is indeed effective. Moreover, it is easy to identify poorly designed exercises since they do not exhibit the characteristics listed above.

Item Type: Book section
Uncontrolled keywords: online assessment, whole sentence input, error diagnosis, generic feedback
Subjects: Q Science > QA Mathematics (inc Computing science) > QA 76 Software, computer programming,
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Computing
Funders: University of Antwerp (
Depositing User: Mark Wheadon
Date Deposited: 24 Nov 2008 18:03 UTC
Last Modified: 12 Jul 2022 10:39 UTC
Resource URI: (The current URI for this page, for reference purposes)
  • Depositors only (login required):

Total unique views for this document in KAR since July 2020. For more details click on the image.