Deriving Common Novice Programming Error Patterns from Student Submissions using LLMs

Abstract

Identifying common difficulties in novice programming is crucial for helping instructors understand where students struggle and for designing targeted interventions. However, extracting actionable insights from large collections of student code submissions typically requires substantial manual review, which can be timeconsuming. In this paper, we investigate whether Large Language Models (LLMs) can derive course-specific summaries of common novice programming error patterns directly from historical code submissions. Results showed that the LLM achieved high accuracy in identifying error patterns and generating useful hints, indicating its potential to support instructor-facing analysis of novice programming difficulties.