Supporting Students' Post-Exam Reflection Needs in College Automation Engineering Course Using LLM
DOI:
https://doi.org/10.58459/icce.2024.4963Abstract
Post-exam reflection is critical in helping students consolidate knowledge acquired during a course, enabling them to apply this understanding in future professional contexts. This study investigates the effectiveness of Mirai, a large language model-based (LLM) chatbot, in supporting students' post-exam reflection needs in an Automation Engineering course. Through a controlled experiment, we explored how context-tuned and non-context-tuned versions of Mirai impacted students' reflection habits, help-seeking behaviors, and perceptions of the tool. Students interact with the chatbot to clarify exam questions and receive personalized explanations. A a surveys based on the extended technology acceptance model (exTAM) was conducted and the resulting data was analyzed. We assessed the efficacy of Mirai in facilitating a deeper understanding of exam-related material, improving students' knowledge, engagement and performance. The findings from this study provide insights into the immediate educational benefits of LLM-based tools, their acceptance among students, and their role in enhancing learning outcomes in engineering education.