Conflow: An Online Concept-Mapping Tool to Measure Conceptual Fluency

Abstract

Assessment at scale operates under constraints of reproducibility, interpretability, and comparability. In practice, these constraints confine pedagogical evaluations to recitations of fact or reproductions of formulaic calculations. Efforts to design novel, more holistic methods of assessment tend to violate one of the three contracts listed above and thus find limited uptake in practice. In this paper, we describe ConFlow - an online assessment instrument that builds on the strengths of concept mapping and aims to satisfy these constraints while enabling holistic, concept-level assessment. We describe the tool and a validation study we undertook to evaluate its performance vis-à-vis personalized ratings provided by college undergraduates. ConFlow lets learners build concept maps, analyze them, and present a ‘quadrant graph’ depicting the mapper’s fluency with the mapped concepts. The quadrant graph categorizes conceptual fluency as ‘Well-Understood’, ‘Partially Understood’, ‘Fuzzy’, and ‘Unknown’ concepts. ConFlow generates this graph based on the number of conceptual moves and the accuracy of the conceptual links and phrases made during the online mapping process. In this pilot work, we present our attempt at validating the tool for wider usage by the learning community. Our process-based assessment method has implications for capturing learning trajectories, and it offers a more profound understanding of concept-level uncertainties and misconceptions.