Marrying Physical and Virtual Realms: An Embodied, Multi-Modal Approach to Situational Learning in Digital Reality

Authors

  • Vando Gusti AL HAKIM Department of Computer Science and Information Engineering, National Central University; Department of Electrical Engineering Education, Universitas Negeri Yogyakarta Author
  • Yao-En CHEN Department of Computer Science and Information Engineering, National Central University Author
  • Meng-Heng LIN Department of Computer Science and Information Engineering, National Central University Author
  • Chia-Ying CHANG Department of Computer Science and Information Engineering, National Central University Author
  • Jen-Hang WANG Department of Computer Science and Information Engineering, National Central University; Research Center for Science and Technology for Learning, National Central University Author
  • Chih-Kai CHANG Department of Information and Learning Technology, National University of Tainan Author
  • YungYu ZHUANG Department of Computer Science and Information Engineering, National Central University Author
  • Su-Hang YANG Department of Hospitality Management, Chien Hsin University of Science and Technology Author
  • Gwo-Dong CHEN Department of Computer Science and Information Engineering, National Central University Author

DOI:

https://doi.org/10.58459/icce.2024.4882

Abstract

In the realm of digital education, traditional virtual reality setups often confine students to basic interactions through devices like a mouse and handheld controllers, limiting immersive learning opportunities, particularly in disciplines requiring multi- modal knowledge such as hospitality and teacher training. Multi-modal knowledge encompasses physical actions, gestures, facial expressions, and other non-verbal communication. This study investigates a novel educational approach that synchronizes physical settings with digital reality scenarios, aided by Al Stereo Cameras (e.g., Apple Vision Pro or MetaQuest), to create a more interactive learning experience. A three-group quasi-experimental design involved 103 Taiwanese students practicing hospitality tasks, such as table service in a simulated Japanese restaurant. The groups included: Group A, handling physical utensils in a real-world-like setting; Group B, controlling virtual utensils with a mouse; and a control group, learning through conventional teaching. Findings reveal that Group A reported significantly better learning outcomes and reduced cognitive load compared to Groups B and C. The reduced cognitive load was attributed to the use of real physical utensils, which minimized the need for students to learn how to navigate virtual utensils using a mouse. The findings underscore the benefits of incorporating physical objects into VR environments, bridging the gap between traditional and digital educational modalities. This approach offers a practical framework for situated learning in educational settings, incorporating embodied and multi-modal learning, and suggesting its broad applicability in improving educational practices and outcomes in the digital age.

Downloads

Download data is not yet available.

Downloads

Published

2024-11-25

How to Cite

Marrying Physical and Virtual Realms: An Embodied, Multi-Modal Approach to Situational Learning in Digital Reality. (2024). International Conference on Computers in Education. https://doi.org/10.58459/icce.2024.4882