Towards Automated Evidence Extraction: A Case Study of Adapting SAM to Real-World Educational Data

Abstract

The demand for a shift from intuition- and experience-based to evidence-based education has been growing. A major challenge in realizing this transition is extracting evidence from real-world educational data. Conventionally, this data extraction process is performed by manually choosing classes for comparison. However, the selection requires expert knowledge of which classes should be compared with which indicators. In this study, we propose the use of deep learning algorithms to uncover inherent causal relationships within vast amounts of existing data. Specifically, we employ structural agnostic modeling, a causal search algorithm known for its exceptional performance on real-world data, to extract complex causal candidates in education. This approach has been referred to as "observational causal discovery." We evaluate the effectiveness of this method using real-world educational data and compare its advantages with those of conventional automatic comparison methods. Results demonstrate that the proposed method can identify various causal candidates, including those that are difficult for humans to discern, and even those without causal relationships. Different from existing methods, the proposed approach does not require selected or fixed comparative indices, thus potentially uncovering comparative indices that elude human comprehension. We anticipate that this research will enable the collection of substantial evidence from real-world educational data and promote evidence-based education.