Extending the Textbook: Co-designing AI-Augmented Interfaces for Mixed Reality Learning in Indian Classrooms

Abstract

Printed textbooks continue to serve as foundational learning resources in many classrooms across the Global South, yet they offer limited support for exploratory, multimodal, or adaptive learning. This paper presents the co-design and prototyping of the INTERACT Suite, a system of AI-augmented interfaces that extends the traditional textbook into a seamless, phygital learning environment. Through a combination of proposed mobile and Mixed Reality-based prototypes named ext-book and interactouch; the system enables learners to interact with textbook content through OCR-based AI explanations, gesture-driven media overlays, and structured annotation workflows. We describe a cooperative design study involving one subject matter expert and six university students, using generative AI-based storyboarding and low-fidelity prototyping to explore interface expectations and cognitive workflows. Key insights include the need for ergonomic design, modular scaffolding, fluid transitions between real and virtual spaces, and context-sensitive multimodal feedback. The study contributes empirically grounded design recommendations for building seamless learning environments that blend physical and metaverse contexts to support learner agency, reflection, and spatial interaction.