An AI-enhanced Pattern Recognition Approach to Analyze Children’s Embodied Interactions

Abstract

This study first presents an approach to the study of computational thinking (CT) as an embodied phenomenon that relies on the crea tion and analysis of multimodal transcripts. The approach, which incorporates a social semiotic approach to multimodality, is then used to train an artificial intelligence (AI) to recognize patterns in the participant’s behaviors that reflect their embodiment of CT during an educational robotics activity. T he AI was developed to ease the labor- intensive aspects of creating and analyzing a multimodal transcript. The findings suggested that the AI -enhanced pattern recognition approach identified similar clusters of activity as human analysis, adding a level of confidence to the analysis of children’s CT that would be difficult to achieve using human analysis.