Observing Facial Muscles to Estimate the Learning State During Collaborative Learning: A Focus on the ICAP Framework

Abstract

This study tests the proposal that a learner’s learning state can be estimated by observing their facial expressions. This could prove helpful for adaptive learning applications using facial recognition technology to provide appropriate feedback to learners. Based on our knowledge of facial expressions and use of the Interactive, Constructive, and Active processes from the ICAP framework, we hypothesized the following: (1) During the Active process, because utterances consist of reading the learning material, the muscles of the mouth are primarily in motion. (2) During the Constructive process, the muscles of the mouth move, and the eyelids tighten, because self-reflection and thoughtful utterances are required. (3) During the Interactive process, the conversation involves conflict; therefore, the eyebrows raise and the eyes open. To that end, we recorded and analyzed the facial expressions and utterances of five pairs of learners during the three learning states. We then organized the data by facial muscles that appeared most frequently during the respective learning states. The analyses generally supported our hypotheses. However, micro-facial expressions generated by facial muscles other than those considered in our hypotheses are also relevant and should be explored in future research.