Logging student answer data in CALL exercises to gauge feedback efficacy

Fowler, Alison M.L. (2006) Logging student answer data in CALL exercises to gauge feedback efficacy. In: 12th International CALL Conference. (Full text available)

Download (174kB) Preview


An important advantage of online assessment work is that answer data can be easily stored and later analysed with a view to establishing the efficacy of the assessment methodology. A 5-year study of the effectiveness of online grammar exercises has been carried out at the University of Kent. The exercises featured require input in the form of whole sentences (since this is a more authentic test of language skills than single word input or multiple choice). Error feedback is generic (indicating where errors have occurred) rather than specific (indicating the exact nature of the errors) because the error-diagnosis system has been designed to be completely language independent. The study aimed to gauge whether this type of feedback is effective - in terms of enabling students to: · identify the types of mistakes in their input; · rectify the mistakes; · learn from the mistakes and apply that learning to subsequent problems There was initial concern that the generic feedback might not provide enough detail to enable users to understand and correct their errors - however extensive use by the University's Spanish department has shown that this type of mark-up is very effective. Chapelle (1998) stresses that it is important for learners to be given the opportunity to correct their linguistic errors. Users of this system, having failed to answer a question correctly on their first attempt, are permitted a second attempt. It is abundantly clear from the logged data that where users make mistakes in their first attempt (and they generally do since the material is designed to be testing), there is almost always a significant improvement in attempt two. This alone would be enough to show that the feedback mode is effective. However this was not enough to prove the pedagogical efficacy of this means of exercise presentation. Therefore more detailed analysis was performed. Over several years of trials more than 100,000 answers have been logged and every answer has been analysed. It can be shown that, for well-designed exercises, as students progress through an exercise they improve in three ways: · more questions are answered correctly on the first attempt; · overall questions scores (i.e. the average of 1<sup>s</sup>t and 2<sup>n</sup>d attempts at questions) improve; · thinking time for formulating answers decreases The degree of increase in accuracy and decrease in thinking time is exercise-dependent, but the overall picture shows clearly that the generic, language-independent feedback is indeed effective. Moreover, it is easy to identify poorly designed exercises since they do not exhibit the characteristics listed above.

Item Type: Conference or workshop item (Paper)
Uncontrolled keywords: online assessment, whole sentence input, error diagnosis, generic feedback
Subjects: Q Science > QA Mathematics (inc Computing science) > QA 76 Software, computer programming,
Divisions: Faculties > Sciences > School of Computing > Computing Education Group
Depositing User: Mark Wheadon
Date Deposited: 24 Nov 2008 18:03 UTC
Last Modified: 24 Apr 2012 14:41 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/14446 (The current URI for this page, for reference purposes)
  • Depositors only (login required):


Downloads per month over past year