Design and Effect of AR Sign Language Teaching Software Based on Natural Interaction

Abstract

Sign language is an important way for deaf people to communicate and learn knowledge. In the process of the promotion and standardization of national common sign language, sign language teaching is becoming more and more important. However, the current sign language teaching is mainly based on multimedia courseware, sign language videos, and sign language action decomposition diagrams, which lack dynamic and all-round display. Therefore, this study focuses on the teaching of alphabetic sign language for young children. Based on platforms and devices such as Unity 3D and Leap Motion, two functions of this sign language teaching software were designed and developed, namely learning function and practice function, and feedback from students, parents and teachers were collected. It enables deaf children and other sign language learners to learn, practice and communicate sign language through Android devices. Finally, the software implements learning function and practice function. Among them, the sign language learning function is intuitive, three-dimensional and interactive, while the practice function has the characteristics of natural interaction and real-time feedback. The average judgment accuracy of the practice function is 90%, calculated through the use effect of eight children, which is relatively high. The interview results show that students, parents and teachers have good feedback on it. They said that although the software needed to be improved, the three-dimensional display of its learning function was very interesting, and the practice function could help students correct their mistakes.