Assessment of Comment Quality in Active Video Watching using Deep Learning

Abstract

This study investigates the effectiveness of deep learning (DL) methods for classifying student comments based on quality within the AVW-Space video-watching platform to enhance student engagement and metacognitive skills. Given the limitations of existing machine learning (ML) techniques, this research explores whether DL models can improve classification performance. The study compared six DL models (BERT, RoBERTa, ALBERT, ELECTRA, GPT, and GPT-2) by training them on a dataset of 13,440 student comments. The results show that RoBERTa outperforms all other models, demonstrating precision, recall, and F1-score improvements. Fine-tuning experiments led to an optimised RoBERTa model. We also examine methods to address class imbalance, with weighted loss functions and random undersampling proving ineffective. Overall, this study contributes to the automation of comment assessment and supports personalised educational experiences, enhancing engagement in video-based learning through interactive commenting.