Extraction of Relationships between Learners’ Physiological Information and Learners’ Mental States by Machine Learning

Abstract

The estimation of learners' mental states during the interaction between teachers and learners is a very important problem in improving the quality of teaching and learning. In this experimental study, we developed a deep learning neural network (DLNN) system that extracted the relationships between a learner’s mental states and a teacher's utterances plus the learner's physiological information. The learner’s physiological information consisted of the NIRS signals, the EEG signals, respiration intensity, skin conductance, and pulse volume. The learner's mental states were elicited through the learner’s introspective reports using the Achievement Emotions Questionnaire (AEQ). According to the AEQ, the learner’s mental states were divided into nine categories: Enjoy, Hope, Pride, Anger, Anxiety, Shame, Hopelessness, Boredom, and Others. In a simulation, the DLNN system exhibited the ability to estimate the learner’s mental states from the learner’s physiological information with high accuracy.