Detecting Off-Task Behavior of Learners in Minecraft Using Exploration and Personalized Features

Abstract

Off-task behavior refers to any action by a learner that is unrelated to the learning task, and it can have a negative effect on learning outcomes. Determining when a behavior is off task is challenging because these behaviors vary across different learning environments and goals. Off-task behavior in Minecraft might be more difficult to detect because of the open-ended nature of the game, which allows learners to explore the environment and complete learning tasks in various ways. This study aims to model off-task behavior of learners using the What-If Hypothetical Implementations using Minecraft (WHIMC). Detector of off-task behavior was developed using features from the interaction logs of the learners in WHIMC. Initially, the detector was constructed using the basic feature set as the baseline, that includes the time-based features, frequency-based features, and sequence features. The feature set was further expanded to include exploration features and personalized features, and the performance of the models using different sets of features were compared. This study found that the detector built with the addition of exploration features to the baseline feature set showed slightly higher performance compared to the detector built using the baseline feature set. Then, the detector built using the feature set that also included personalized features had better performance compared to the detector using the feature set with the exploration features. Some selected exploration and personalized features were also found to be more predictive of off-task behavior compared to some features in the baseline feature set.