Analytics of Open-Book Exams with Interaction Traces in a Humanities Course

Abstract

Open book exams (OBE) have been a mandated part of each course structure at some universities. Also during the COVID19 emergency remote teaching situation, OBE would be an option for many instructors over a proctored examination. In this study we investigate a Critical Analysis of Literature and Cinema course which had offered open book exam components for more than 11 years in a face-to-face classroom mode. However, this time the OBE was conducted online using BookRoll, a learning analytics enhanced eBook platform. 89 Students accessed Hayavadana, an Indian play uploaded on BookRoll during the exam. They attempted a critical reading task to identify performative elements and cultural references in the text by highlighting them with yellow and red markers respectively and writing a reflective memo about the identified items in BookRoll. We analyzed learner’s interaction logs gathered in the learning record store linked to BookRoll during the OBE and investigated the relations between their critical reading behaviors to the OBE achievement. Further, selecting two distinct achievement groups we conducted process mining to identify distinct reading behaviors of the high and low performers and give examples of their generated reflective memos. This study aims to initiate further discussion related to the application of learning analytics in humanities courses and probed into the behaviors of the students during the OBE.