Marrying Physical and Virtual Realms: An Embodied, Multi-Modal Approach to Situational Learning in Digital Reality
DOI:
https://doi.org/10.58459/icce.2024.4882Abstract
In the realm of digital education, traditional virtual reality setups often confine students to basic interactions through devices like a mouse and handheld controllers, limiting immersive learning opportunities, particularly in disciplines requiring multi- modal knowledge such as hospitality and teacher training. Multi-modal knowledge encompasses physical actions, gestures, facial expressions, and other non-verbal communication. This study investigates a novel educational approach that synchronizes physical settings with digital reality scenarios, aided by Al Stereo Cameras (e.g., Apple Vision Pro or MetaQuest), to create a more interactive learning experience. A three-group quasi-experimental design involved 103 Taiwanese students practicing hospitality tasks, such as table service in a simulated Japanese restaurant. The groups included: Group A, handling physical utensils in a real-world-like setting; Group B, controlling virtual utensils with a mouse; and a control group, learning through conventional teaching. Findings reveal that Group A reported significantly better learning outcomes and reduced cognitive load compared to Groups B and C. The reduced cognitive load was attributed to the use of real physical utensils, which minimized the need for students to learn how to navigate virtual utensils using a mouse. The findings underscore the benefits of incorporating physical objects into VR environments, bridging the gap between traditional and digital educational modalities. This approach offers a practical framework for situated learning in educational settings, incorporating embodied and multi-modal learning, and suggesting its broad applicability in improving educational practices and outcomes in the digital age.