Proposal and Evaluation of a Method of Estimating the Difficulty of Items Based on Item Types and Similarity of Choices

Abstract

In recent years, on the idea of supporting the composition of the tests by using statistical data, such as the difficulty level of the items that constitute the tests, has been investigated. In general, item response theory (IRT) is used to quantify the difficulty level of items. However, this approach requires that the items are answered by many learners in advance and it is difficult to ensure that all items in the bank are answered. We propose a method of estimating the difficulty level of unanswered items. In our method, the level of new items is estimated from the level of similar existing items based on the differences between item types and the similarity between choices. A simulation experiment shows that the difficulty levels calculated by IRT and by using the proposed method can have a reasonable correlation. However the results obtained using the new estimation method can be very different from the IRT results if incorrect answers to an item are similar to the correct answer.