Effect of an Automated Writing Evaluation System on Students’ EFL Writing Performance

Abstract

Recently, a growing number of studies were conducted on the use of automated writing evaluation (AWE) systems in writing classrooms. As an online writing evaluation system, Pigai has been used more and more widely in college-level English as a Foreign Language (EFL) writing courses. The current study examines the accuracy of the feedback from Pigai and the effect of the AWE system on Chinese college students’ EFL writing performance. 97 students from an academic writing course were invited to participate in the study. This study addresses the following two research questions: 1) How accurate is the feedback from Pigai in terms of the errors it can identify in students’ writing? 2) What is the effect of using the AWE system on students’ essay performance? Results showed that the average precision rate of Pigai is 58% and the accuracy varies across error types. In addition, Pigai improved students’ writing performance significantly in terms of the quality and length of essays. The findings will inform discussion of whether and how to integrate the use of online writing evaluation systems into writing classrooms.