Automated Multilingual Sentiment Analysis of Student Comments in Faculty Evaluations using Transformer-Based AI

Abstract

Open-ended student comments in faculty evaluations offer rich evidence for improving teaching, but the scale, subjectivity, and frequent code-mixing (English–Tagalog–Bisaya) make manual analysis slow and inconsistent. This research presents ClasSentiments, an automated, human-in-the-loop pipeline for multilingual sentiment analysis in higher education. Using a five-year dataset of 24,000 comments from a Philippine state university, we fine-tune and compare three transformer models—mBERT, Twitter-RoBERTa-base, and GPT-2—on a balanced training set (9,129 comments; 3,043/class). Twitter-RoBERTa achieves the best test performance (88% accuracy and the highest macro-F1) and generalizes on an expert-verified holdout (91.3% overall on 300 comments). We deploy the best model in a lightweight web application that provides per-comment labels, aggregate visualizations, and manual overrides with audit logs to preserve human judgment in high-stakes decisions. A formative usability study with staff, chairs, and instructors yields a SUS score of 87 (excellent). Contributions include (1) an empirical comparison of transformer architectures for short, code-mixed educational feedback; (2) a deployable analytics tool that integrates model outputs with human oversight; and (3) evidence of real-world readiness via expert validation and usability results. The approach aligns with learning analytics goals by turning qualitative student voice into timely and actionable insights for instructional improvement.