Diagnostic Language Assessment: Lessons Learned from Rapid Prototyping

Abstract

In this paper, we provide the rationale for a computerized diagnostic language assessment (DLA) of email writing for Japanese undergraduates. The development of a rapid prototype of the DLA system is then described, and lessons learned from the process are shared. The DLA focuses on pragmatic competence – the ability to alter one’s language choices relating to politeness, formality and structure, depending on the social context of the communication. While tools now exist to assist with learners’ grammar problems and vocabulary, few exist for this pragmatic aspect of communication. Furthermore, traditional language assessments typically provide an overall score of learner ability without identifying specific strengths and weaknesses. The DLA seeks to address this issue via a three-stage process: 1) assessment administration; 2) provision of feedback to the learner, and 3) provision of further individualized instruction based on the assessment results. To gain a better perspective of the problems and to start to climb the learning curve as quickly as possible, rapid prototyping using Axure RP was selected. This high-fidelity prototype of the DLA was designed for some use cases. Numerous lessons were learned. Notably, rapid prototyping is not necessarily rapid. Early feedback on usability and the user experience were received from the rapid prototyping, helping the developer gain a better understanding of the user needs and discover any unexpected usability issues prior to actual coding. Another ongoing advantage is that rather than describing the expected functionalities of the software to stakeholders or interested parties, the functionalities of the fully-fledged DLA can be shown using the prototype. The prototype functions as an interactive visual aid, reducing miscommunication and providing a focus around which discussions can be based.