The Effect of Feedback in Chatbot-based Pre-class Learning Environment

Abstract

With the rapid advancement of large language models (LLMs), chatbots powered by these models have recently been introduced to support pre-class learning. However, a potential concern is that students may develop misunderstandings during their interaction with the chatbot. To address this issue, incorporating exercises and feedback after chatbot-based learning is considered effective. There are two main types of feedback: Knowledge of Correct Response (KCR), and Explanation Feedback (ExF). Each type is believed to have different effects on learning outcomes. However, no research has compared these effects in the context of chatbot-based pre-class learning. Therefore, in this study we developed LLM-based Chatbot and Feedback-Tool to promote early awareness of misunderstandings and deepen understanding. Additionally, we evaluated the effectiveness of the developed learning support tools, as well as the effect of KCR and ExF on learning outcomes. The experimental results demonstrated the effectiveness of the learning support tools in the pre-class learning. Furthermore, it showed no significant difference in learning outcomes between the two types of feedback. However, students perceived ExF as more useful.