Learning Analysis infrastructure to support optimal learning based on IRT and Video Enrollment Data

Abstract

Currently, local governments in Japan need DX (=Digital Transformation) promotion personnel. However, even today, Japan's civil service system still employs mainly legal and human personnel, resulting in a need for more necessary human resources. Therefore, the demand for DX human resource development for local government officials is rapidly increasing. In response to this, we have been engaged in human resource development using e-learning materials based on Moodle for three years since 2020, targeting approximately 9,500 employees of the Hiroshima City Office, where I work, and through formative evaluation of the materials, we have succeeded in creating materials at a level where all employees who took the course expressed a desire to promote DX. The company succeeded in developing educational materials so that all staff members who took the course were willing to promote DX. However, only about 5% of all staff members completed the course, reaching the limit of the improvement method in formative evaluation from those who completed the course. During the three-year effort, we also identified a wide gap in the IT skills of local officials and that we cannot provide optimal educational materials for 95% of the staff. In this study, we created multiple-choice questions corresponding to the DX literacy standard set by the Ministry of Economy, Trade and Industry (METI) in December 2022, which we regarded as a complete mastery map, and optimized them using IRT. The objective was to develop a model for measuring the DX literacy of local government officials and an e-learning course that provides individualized and optimal learning based on the measurement results. Through these efforts, we would like to establish a method of correlating skill surveys using IRT (=Item Response Theory) with e-learning course attendance data to overcome the limitations of the technique of creating e-learning courses and improving them through formative evaluation and to present a method to overcome the limitations of formative assessment in the ID (=Instructional Design) field. We hope to offer a strategy to overcome the limitations of formative evaluation in ID.