Design and Preliminary Evaluation of an AI-Supported Role Fulfillment System: A Case Study on Delayed Skill Transfer in Collaborative Learning

Abstract

While Computer-Supported Collaborative Learning (CSCL) systems effectively structure group interactions, their impact on long-term skill internalization remains underexplored. This exploratory case study examines an AI-supported role fulfillment system that provides real-time, role-specific feedback to individual learners via a low-latency streaming pipeline and LLM-based analysis. Using a within-subjects reversal design (Pre-Test, AI-supported Test, Post-Test) with four university students, we observed two complementary transfer effects: a delayed equalization in participation (44.2% reduction in word-based Gini coefficient only after AI removal) and sustained role fulfillment (+19.3% during intervention, +17.0% maintained postintervention). These temporally dissociated patterns suggest that a single brief AI intervention can simultaneously support implicit norm acquisition and explicit skill development, though observable effects emerge on different timescales. While limited by sample size, these preliminary findings highlight the importance of post-intervention assessment in CSCL research.