Tackling Unserious Raters in Peer Evaluation: Behavior Analysis and Early Detection with Learner Model

Abstract

Peer evaluation of individual or group work is often adopted in team-based learning design. However, some raters may not take the evaluation process seriously and exhibit behaviors such as using the same score, rushing through evaluations, or not evaluating during the presentation. This study investigates the issue of unserious peer evaluation in group presentations, focusing on their behavior patterns. Using evaluation behavior analysis indicators, we identified unserious raters who exhibited low reliability in the peer evaluation process. Further, we conducted a preliminary analysis to detect unserious raters based on learner model data available before the peer evaluation process. This information can assist teachers in providing personalized prompts and interventions prior to the peer evaluation process, thus enhancing the evaluation quality of these students with timely prompts to them.