Privacy-Focused AI Avatar for Educational Robotics in the Metaverse

Abstract

The convergence of immersive environments and educational robotics is transforming STEM learning by enabling students to design and deploy robots across both virtual and physical contexts. This paper presents a privacy-first conversational AI avatar that enables seamless voice interaction with robots in simulation and reality. All processing runs locally, safeguarding student data while ensuring real-time responsiveness. The system integrates Whisper for speech recognition, a lightweight Llama 3.2 model for intent understanding, and Glow-TTS for natural feedback. Robot commands are strictly validated against a JSON schema and executed via ROS 2, allowing seamless transfer between NVIDIA Isaac Sim and a Jetson Orin Nano-powered robot. A prototype workshop confirmed smooth interaction, safe command execution, and high engagement, highlighting potential for immersive, privacy-preserving robotics education.