Empowering Children to Create AI-Enabled Augmented Reality Experiences
Not on the map. Why?
Abstract (arXiv)
Despite their potential to enhance children's learning experiences, AI-enabled AR technologies are predominantly used in ways that position children as consumers rather than creators. We introduce Capybara, an AR-based and AI-powered visual programming environment that empowers children to create, customize, and program 3D characters overlaid onto the physical world. Capybara enables children to create virtual characters and accessories using text-to-3D generative AI models, and to animate these characters through auto-rigging and body tracking. In addition, our system employs vision-based AI models to recognize physical objects, allowing children to program interactive behaviors between virtual characters and their physical surroundings. We demonstrate the expressiveness of Capybara through a set of novel AR experiences. We conducted user studies with 20 children in the United States and Argentina. Our findings suggest that Capybara can empower children to harness AI in authoring personalized and engaging AR experiences that seamlessly bridge the virtual and physical worlds.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 92 references that are in the map. First posted on arXiv 11 Aug 2025.
Builds on
24 of 92 references.
References · 24
- 2023Promptify: Text-to-Image Generation through Interactive Prompt Exploration with Large Language Modelscited by 67
- 2024PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinementcited by 62
- 2022Large-scale Text-to-Image Generation Models for Visual Artists' Creative Workscited by 61
- 2023LLMR: Real-time Prompting of Interactive Worlds using Large Language Modelscited by 39
- 2022StoryBuddy: A Human-AI Collaborative Chatbot for Parent-Child Interactive Storytelling with Flexible Parental Involvementcited by 37
- 2020RigNet: Neural Rigging for Articulated Characterscited by 32
- 2024Mathemyths: Leveraging Large Language Models to Teach Mathematical Language through Child-AI Co-Creative Storytellingcited by 21
- 2020RealitySketch: Embedding Responsive Graphics and Visualizations in AR through Dynamic Sketchingcited by 14
- 2022RealityTalk: Real-Time Speech-Driven Augmented Presentation for AR Live Storytellingcited by 10
- 2024VRCopilot: Authoring 3D Layouts with Generative AI Models in VRcited by 10
- 2023Teachable Reality: Prototyping Tangible Augmented Reality with Everyday Objects by Leveraging Interactive Machine Teachingcited by 7
- 2023Augmented Math: Authoring AR-Based Explorable Explanations by Augmenting Static Math Textbookscited by 7
12 more
- 2024ChatScratch: An AI-Augmented System Toward Autonomous Visual Programming Learning for Children Aged 6-12cited by 7
- 2025LLMs and Childhood Safety: Identifying Risks and Proposing a Protection Framework for Safe Child-LLM Interactioncited by 7
- 2022Project IRL: Playful Co-Located Interactions with Mobile Augmented Realitycited by 6
- 2024How People Prompt to Create Interactive VR Scenescited by 6
- 2024Augmented Physics: Creating Interactive and Embedded Physics Simulations from Static Textbook Diagramscited by 6
- 2024Children's Mental Models of Generative Visual and Text Based AI Modelscited by 3
- 2025From Following to Understanding: Investigating the Role of Reflective Prompts in AR-Guided Tasks to Promote Task Understandingcited by 3
- 2016Remixing as a Pathway to Computational Thinkingcited by 2
- 2017Autocomplete 3D Sculptingcited by 2
- 2021SceneAR: Scene-based Micro Narratives for Sharing and Remixing in Augmented Realitycited by 1
- 2022Auggie: Encouraging Effortful Communication through Handcrafted Digital Experiencescited by 1
- 2024Understanding Young People's Creative Goals with Augmented Realitycited by 1
Built on by
Not cited yet.