MESHing Minds: Bridging the Gap Between Creativity and IoT Programming Through Collaborative Mixed Reality

Abstract

Fostering creativity in programming tasks is a challenging endeavor, especially when working alone. Traditional programming environments often lack support for stimulating creative thinking and idea generation. This paper presents an interactive augmented reality (AR) system that aims to enhance creativity in Internet of Things (IoT) programming tasks. The proposed system leverages Interactive Evolutionary Computation (IEC) and AR technologies to facilitate the collaborative exploration and evolution of IoT device programs (MESH programs). Users can create, evaluate, and iteratively refine MESH programs through an immersive AR interface, while being inspired by the system's suggestions and other users' creations. The system employs a genetic algorithm to evolve MESH programs based on user evaluations, and utilizes natural language processing to generate program descriptions that can trigger new ideas. A user study (n=16) was conducted to evaluate the system's effectiveness in stimulating creativity and promoting collaboration. Quantitative analysis revealed a significant increase in idea generation over time and a greater impact on inspiration for novice programmers. Qualitative findings highlighted the system's ability to foster a creative and collaborative environment. The insights gained from this study inform the design of future tools and experiences that support creative thinking and collaboration in the IoT domain.