Unlocking student voices: using large language models to analyse open-ended survey responses at scale

Abstract

This paper overviews a project using large language models (LLMs) to analyse open-ended responses from a student learning survey. Traditional qualitative analysis is time-consuming, but LLMs can quickly process large volumes of text while preserving detail. The project applies LLMs to classify diverse student comments, aiding understanding of student experiences and improving reporting speed. This reduces manual effort and supports timely, evidence-based decisions. The paper also explores practical insights and challenges encountered, including prompt optimisation and the validation of AI-generated classifications to support meaningful insights for targeted follow-up actions.