Process Evaluation for Concept Map Building and Its Experimental Evaluation

Abstract

Concept map as an assessment tool is widely accepted in the field of education. In this research, assuming the process evaluation of concept map building leads to better evaluation than the product (that is, built concept map) evaluation, we proposed a process evaluation method and conducted an experimental evaluation of the method. The process evaluation method is intended for re-composition type of concept maps, not for general ones. In the re-composition concept map, a learner is provided a set of components of a concept map, that is, links and nodes, and required to recompose a concept map by using the components. The provided components are generated by decomposing the concept map built by an expert as a representation of understanding of the learning target. In this research, we set several submaps with a set of semantics in one expert map and define a good process by recomposing the submap as a series of activities. The process evaluation results were compared with the product evaluation results for the data of a practical use of re-composition of concept map in undergraduate class. As the results of the comparison, we have already confirmed that the result of process evaluation was a better predictor of learner’s understanding than the results of the product evaluation. In this paper as an additional analysis, we reported the analysis of relation between process evaluation and product evaluation. The difference between the evaluation methods could result in a case where there is zero score for process evaluation and full score for product evaluation. However, analysis results confirmed that there is a strong correlation between the two evaluation methods with r value = 0.8. These results suggest that support for the process of submap re-composition is a promising way for learners to recompose a good concept map. We further explore the usability of the process evaluation to the teachers, and it shows a high rate of agreement of the usefulness and appropriateness of the method. Based on the results, we are developing a formative assessment and support environment for concept map re-composition.