The Effects of Difficulty-Based Question Adaptivity in Formative Assessment on Introductory Programming Learning

Abstract

Introductory programming is an essential component of computer science education, yet many students find it challenging. An important factor in students' success in programming courses is their motivation and self-confidence. Numerous formative assessment systems have been designed to increase students' self-confidence. These systems are capable of automatically evaluating student work and delivering immediate feedback. Students gain confidence as a result of formative assessment's automatic feedback, which enables them to better evaluate their own work and make improvements. Many formative assessment systems ask the same questions of students with varying skill levels. Therefore, the objectives of this paper are: first, to introduce an adaptive strategy in formative assessment that poses the questions using difficulty levels; second, to determine whether this adaptive formative assessment framework can help students feel more confident by introducing common code errors, and third, to evaluate how well students performed following an adaptive formative assessment activity. An experimental study with 62 undergraduate Non-CS participants (novices) was conducted to accomplish these objectives. In-depth analyses were performed on the self-rated confidence, test results and completion time. According to the findings, novices feel significantly better confident after taking the adaptive formative assessment quizzes. However, the self-rated confidence declines over the advanced topics of programming. Additionally, students who took part in the revision adaptive formative assessment exercise performed better on their test. Findings showed that employing difficulty-based question adaptivity in formative assessment was more likely to motivate students and increase their self-confidence.