Identifying Cognitive and Metacognitive Processes Using Retrospective and Concurrent Think-Aloud Protocols

Abstract

Think-aloud protocols - Retrospective Think-Aloud (RTA) and Concurrent Think-Aloud (CTA) - are widely used in educational research to examine cognitive and metacognitive processes. RTA captures reflective verbalizations after task completion, often yielding elaborate explanations, while CTA records real-time thoughts, offering direct insights into learners’ in-task thinking. Although both protocols uncover complex learning processes, CTA is more commonly used to model metacognition. This study compares RTA and CTA in identifying cognitive and metacognitive processes during problem-solving in MEttLE (Modelling-based Estimation Learning Environment), a computer-based, open-ended environment. We conducted parallel studies using both protocols as learners engaged in estimation problem-solving tasks. The analysis focused on: (1) the overall frequency of verbalizations, (2) the frequency of cognitive and metacognitive processes, and (3) a temporal analysis of metacognitive activity using CTA. Findings show that CTA yielded more on-task verbalizations than RTA. RTA captured high-level control and reflective processes such as model building, estimation reasoning, and adaptive attributions. In contrast, CTA revealed procedural strategies like selecting and adapting learning approaches and gathering context-specific knowledge. CTA also enabled temporal tracking of metacognitive shifts throughout the task. These findings highlight the complementary strengths of both protocols and offer methodological insights for researchers seeking to analyze learners’ thinking in technology-enhanced learning environments.