Reliability of Generative AI Based Reflection Chatbot Using the 5Rs Model

Abstract

This study examines the reliability of a generative AI based reflection chatbot using the 5Rs model with dialogue logs from student employees. Results show moderate agreement but strong consistency in relative ordering. Discrepancies reflect interpretive differences, suggesting that reflection-level estimates function as internal signals for structuring dialogue and that low-cost models are sufficient in this purpose.