Automatic Classification of MOOC Forum Messages to Measure the Quality of Peer Interaction

Abstract

Discussion forum is an integral part of many MOOCs as it provides a platform for peer interaction among learners. The quality of peer interaction is an indicator of the potential for peer learning. Thus, quality of peer interaction provides instructors with an actionable insight into the extent of critical or higher level thinking that learners are engaged in and is a measure of the learning effectiveness of the course. It is daunting for instructors to manually analyze the forum messages to gain this insight. To address this issue, we attempted to develop a system for automatic classification of forum messages that will inform instructors on the quality of peer interaction happening in the forum. Our system classifies messages into predefined classes based on the Interaction Analysis Model phases. We explored and implemented multiple machine learning models. A general accuracy of 95%-97% was observed among the models and no model outperformed the other by a great margin. The need for such a system has become all the more relevant in the current Covid-19 pandemic situation, where all physical classrooms have had to migrate to an online setting.