GenAI Voice Mode in Programming Education
Not on the map. Why?
Abstract (arXiv)
Real-time voice interfaces using multimodal Generative AI (GenAI) can potentially address the accessibility needs of novice programmers with disabilities (e.g., related to vision). Yet, little is known about how novices interact with GenAI tools and their feedback quality in the form of audio output. This paper analyzes audio dialogues from nine 9th-grade students using a voice-enabled tutor (powered by OpenAI's Realtime API) in an authentic classroom setting while learning Python. We examined the students' voice prompts and AI's responses (1210 messages) by using qualitative coding. We also gathered students' perceptions via the Partner Modeling Questionnaire. The GenAI Voice Tutor primarily offered feedback on mistakes and next steps, but its correctness was limited (71.4% correct out of 416 feedback outputs). Quality issues were observed, particularly when the AI attempted to utter programming code elements. Students used the GenAI voice tutor primarily for debugging. They perceived it as competent, only somewhat human-like, and flexible. The present study is the first to explore the interaction dynamics of real-time voice GenAI tutors and novice programmers, informing future educational tool design and potentially addressing accessibility needs of diverse learners.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 52 references that are in the map. First posted on arXiv 12 Sep 2025.
Builds on
23 of 52 references.
References · 23
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 216
- 2023The Robots are Here: Navigating the Generative AI Revolution in Computing Educationcited by 85
- 2022Experiences from Using Code Explanations Generated by Large Language Models in a Web Software Development E-Bookcited by 84
- 2024CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needscited by 81
- 2023CodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classescited by 71
- 2023"It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmerscited by 62
- 2024The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmerscited by 46
- 2023Exploring the Potential of Large Language Models to Generate Formative Programming Feedbackcited by 28
- 2023User Interaction Patterns and Breakdowns in Conversing with LLM-Powered Voice Assistantscited by 14
- 2024Beyond the Hype: A Comprehensive Review of Current Trends in Generative AI Research, Teaching Practices, and Toolscited by 13
- 2024Student-AI Interaction: A Case Study of CS1 studentscited by 11
11 more
- 2024How Novice Programmers Use and Experience ChatGPT when Solving Programming Exercises in an Introductory Coursecited by 11
- 2024Evaluating the Application of Large Language Models to Generate Feedback in Programming Educationcited by 10
- 2024Analyzing Chat Protocols of Novice Programmers Solving Introductory Programming Tasks with ChatGPTcited by 8
- 2024Leveraging Lecture Content for Improved Feedback: Explorations with GPT-4 and Retrieval Augmented Generationcited by 6
- 2024You're (Not) My Type -- Can LLMs Generate Feedback of Specific Types for Introductory Programming Tasks?cited by 6
- 2025Unlimited Practice Opportunities: Automated Generation of Comprehensive, Personalized Programming Taskscited by 4
- 2025From Prompts to Propositions: A Logic-Based Lens on Student-LLM Interactionscited by 4
- 2025The Role of Generative AI in Software Student CollaborAItioncited by 2
- 2025That's Not the Feedback I Need! -- Student Engagement with GenAI Feedback in the Tutor Kaicited by 2
- 2024Cross-Cultural Validation of Partner Models for Voice User Interfacescited by 1
- 2025Students' Feedback Requests and Interactions with the SCRIPT Chatbot: Do They Get What They Ask For?cited by 1
Built on by
Not cited yet.