ALBe: An AI Literacy Behavioral Assessment Framework Based on EFL Chatbot Interaction Logs

Abstract

The accelerating integration of artificial intelligence (AI) technology into education and its close connection with academic performance are leading to increasing awareness of supporting learners’ AI literacy. However, the existing instruments lack evidence-based assessment of AI literacy. In this study, we present the AI Literacy Behavioral Framework (ALBe), a codebook instrument developed following established procedures in behavioral measurement of student communication with Tammy. After human labeling, elbow and silhouette were processed to confirm the K-means (k=3) to classify students into developing, emerging, and advanced-level groups based on three dimensions of AI literacy. These results suggest that ALBe offers a potential and valid method for assessing AI literacy.