mlsanityData to 14 Oct 2026

Students' Feedback Requests and Interactions with the SCRIPT Chatbot: Do They Get What They Ask For?

arXiv 2507.17258 ↗First posted 23 Jul 2025cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Building on prior research on Generative AI (GenAI) and related tools for programming education, we developed SCRIPT, a chatbot based on ChatGPT-4o-mini, to support novice learners. SCRIPT allows for open-ended interactions and structured guidance through predefined prompts. We evaluated the tool via an experiment with 136 students from an introductory programming course at a large German university and analyzed how students interacted with SCRIPT while solving programming tasks with a focus on their feedback preferences. The results reveal that students' feedback requests seem to follow a specific sequence. Moreover, the chatbot responses aligned well with students' requested feedback types (in 75%), and it adhered to the system prompt constraints. These insights inform the design of GenAI-based learning support systems and highlight challenges in balancing guidance and flexibility in AI-assisted tools.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 27 of its 54 references that are in the map. First posted on arXiv 23 Jul 2025.

Builds on

27 of 54 references.

References · 27

  1. 2022Automatic Generation of Programming Exercises and Code Explanations using Large Language Modelscited by 126
  2. 2023The Robots are Here: Navigating the Generative AI Revolution in Computing Educationcited by 86
  3. 2022Experiences from Using Code Explanations Generated by Large Language Models in a Web Software Development E-Bookcited by 84
  4. 2024CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needscited by 84
  5. 2023CodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classescited by 72
  6. 2022Using Large Language Models to Enhance Programming Error Messagescited by 67
  7. 2023Generating High-Precision Feedback for Programming Syntax Errors using Large Language Modelscited by 34
  8. 2023Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Codecited by 32
  9. 2023Exploring the Potential of Large Language Models to Generate Formative Programming Feedbackcited by 28
  10. 2024Evaluating the Effectiveness of LLMs in Introductory Computer Science Education: A Semester-Long Field Studycited by 26
  11. 2023Next-Step Hint Generation for Introductory Programming Using Large Language Modelscited by 20
  12. 2023Transformative Effects of ChatGPT on Modern Education: Emerging Era of AI Chatbotscited by 18
15 more
  1. 2024Feedback-Generation for Programming Exercises With GPT-4cited by 14
  2. 2024Beyond the Hype: A Comprehensive Review of Current Trends in Generative AI Research, Teaching Practices, and Toolscited by 14
  3. 2023Dcc --help: Generating Context-Aware Compiler Error Explanations with Large Language Modelscited by 11
  4. 2024How Novice Programmers Use and Experience ChatGPT when Solving Programming Exercises in an Introductory Coursecited by 11
  5. 2023Large Language Models in Introductory Programming Education: ChatGPT's Performance and Implications for Assessmentscited by 10
  6. 2024Evaluating the Application of Large Language Models to Generate Feedback in Programming Educationcited by 10
  7. 2024Exploring How Multiple Levels of GPT-Generated Programming Hints Support or Disappoint Novicescited by 10
  8. 2023Can ChatGPT Pass An Introductory Level Functional Language Programming Course?cited by 9
  9. 2024Analyzing Chat Protocols of Novice Programmers Solving Introductory Programming Tasks with ChatGPTcited by 8
  10. 2024You're (Not) My Type -- Can LLMs Generate Feedback of Specific Types for Introductory Programming Tasks?cited by 6
  11. 2025Unlimited Practice Opportunities: Automated Generation of Comprehensive, Personalized Programming Taskscited by 4
  12. 2024LLM-Driven Feedback for Enhancing Conceptual Design Learning in Database Systems Coursescited by 2
  13. 2025The Role of Generative AI in Software Student CollaborAItioncited by 2
  14. 2025Open, Small, Rigmarole -- Evaluating Llama 3.2 3B's Feedback for Programming Exercisescited by 2
  15. 2025That's Not the Feedback I Need! -- Student Engagement with GenAI Feedback in the Tutor Kaicited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025GenAI Voice Mode in Programming Educationcited by 0