A Machine Learning Approach to Estimating Student Mastery by Predicting Feedback Request and Solving Time in Online Learning System

Abstract

One of the most significant challenges for computers in education is the capacity to provide intelligent and adaptable learning systems to meet the real needs of students. In order to create efficient adaptive or personalized mechanisms for educational content, student models are proposed to estimate the actual knowledge or mastery level of students. Some earlier student models were proposed to estimate student mastery based on the correctness (e.g., correct or incorrect) of responses, feedback request, and solving time using classical Markov process and logistic regression models. In particular, these models were applied to predicting student future correctness, feedback request, and solving time (i.e., on the next question).The advent of increasingly large-scale datasets has turned Machine Learning (ML) methods such as conventional machine-learning algorithms and deep learning models for prediction into competitive alternatives to classical Markov process and logistic regression models. In addition, prediction by ML methods has numerous advantages such as interpretability, good accuracy, ease of maintenance, less execution time, and appropriately handling of missing data. Moreover, recent studies exhibit the significant achievement of ML prediction methods for estimating students' performance and mastery using learning log data (i.e., correctness, feedback request level, solving time, etc.). Hence, it is reasonable to use ML methods to estimate student mastery by predicting the feedback request level and solving time. This study analyzed the data logged by an online learning system called Math-Island, which teaches elementary level mathematics by incorporating game mechanisms and scaffolding feedback. Machine-learning regression methods such as Multiple Linear Regression (MLR), Support Vector Regression (SVR), Random Forest Regression (RFR), Extra Trees (ET), and Gradient Boosting Regression (GBR) were applied. The results showed that RFR and GBR were found to outperform other models to predict future feedback request level and solving time. The results lead to several future works. First, incorporating ML predictive models into Math-Island tutoring system to identify the individual student's actual needs and; reduce learning loss substantially. Second, it drives to effectively build a more efficient adaptive mechanism within the current session to utilize students’ active learning time.