A Deep-Learning Technique for Converting Bengali Handwritten Answer-Book Images into Text

Abstract

Computer-Assisted Assessment (CAA) involves using computer technology for educational assessment. Handwritten answer books are the primary medium for various levels of educational assessment in schools in India. Handwritten answer books pose the key challenge in adopting CAA in the Indian language, Bengali. These answer books must be converted into machine-readable text before any tool is used to evaluate the answers. The literature on Bengali handwritten word recognition primarily focused on considering the image of a single word as input. However, for automatic answer evaluation, the image of the whole page should be taken as input by the system. This paper aims to develop a system for converting handwritten words from a student's answer book into machine-readable text. The proposed system follows a two-phase architecture. The first phase automatically segments the words using a Mask Region-based Convolutional Neural Network (R-CNN) network. Then, the segmented words are used as input for the second phase of image-to-text conversion using a CNN-Transformer network. An openly available dataset and an in-house dataset of answer books collected from Bengali medium schools are used to train the models. The experimental results show that the proposed model is quite promising.