# Students' Feedback Requests and Interactions with the SCRIPT Chatbot: Do They Get What They Ask For?

arXiv: https://arxiv.org/abs/2507.17258 · HTML: https://mlsanity.com/paper/2507.17258/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 54 references that are in the map.
- First posted on arXiv 23 Jul 2025.

## Builds on (most cited first, 27 of 27)

- [Automatic Generation of Programming Exercises and Code Explanations using Large Language Models](https://mlsanity.com/paper/2206.11861/) (2022)
- [The Robots are Here: Navigating the Generative AI Revolution in Computing Education](https://mlsanity.com/paper/2310.00658/) (2023)
- [Experiences from Using Code Explanations Generated by Large Language Models in a Web Software Development E-Book](https://mlsanity.com/paper/2211.02265/) (2022)
- [CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needs](https://mlsanity.com/paper/2401.11314/) (2024)
- [CodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classes](https://mlsanity.com/paper/2308.06921/) (2023)
- [Using Large Language Models to Enhance Programming Error Messages](https://mlsanity.com/paper/2210.11630/) (2022)
- [Generating High-Precision Feedback for Programming Syntax Errors using Large Language Models](https://mlsanity.com/paper/2302.04662/) (2023)
- [Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Code](https://mlsanity.com/paper/2303.08033/) (2023)
- [Exploring the Potential of Large Language Models to Generate Formative Programming Feedback](https://mlsanity.com/paper/2309.00029/) (2023)
- [Evaluating the Effectiveness of LLMs in Introductory Computer Science Education: A Semester-Long Field Study](https://mlsanity.com/paper/2404.13414/) (2024)
- [Next-Step Hint Generation for Introductory Programming Using Large Language Models](https://mlsanity.com/paper/2312.10055/) (2023)
- [Transformative Effects of ChatGPT on Modern Education: Emerging Era of AI Chatbots](https://mlsanity.com/paper/2306.03823/) (2023)
- [Feedback-Generation for Programming Exercises With GPT-4](https://mlsanity.com/paper/2403.04449/) (2024)
- [Beyond the Hype: A Comprehensive Review of Current Trends in Generative AI Research, Teaching Practices, and Tools](https://mlsanity.com/paper/2412.14732/) (2024)
- [Dcc --help: Generating Context-Aware Compiler Error Explanations with Large Language Models](https://mlsanity.com/paper/2308.11873/) (2023)
- [How Novice Programmers Use and Experience ChatGPT when Solving Programming Exercises in an Introductory Course](https://mlsanity.com/paper/2407.20792/) (2024)
- [Large Language Models in Introductory Programming Education: ChatGPT's Performance and Implications for Assessments](https://mlsanity.com/paper/2308.08572/) (2023)
- [Evaluating the Application of Large Language Models to Generate Feedback in Programming Education](https://mlsanity.com/paper/2403.09744/) (2024)
- [Exploring How Multiple Levels of GPT-Generated Programming Hints Support or Disappoint Novices](https://mlsanity.com/paper/2404.02213/) (2024)
- [Can ChatGPT Pass An Introductory Level Functional Language Programming Course?](https://mlsanity.com/paper/2305.02230/) (2023)
- [Analyzing Chat Protocols of Novice Programmers Solving Introductory Programming Tasks with ChatGPT](https://mlsanity.com/paper/2405.19132/) (2024)
- [You're (Not) My Type -- Can LLMs Generate Feedback of Specific Types for Introductory Programming Tasks?](https://mlsanity.com/paper/2412.03516/) (2024)
- [Unlimited Practice Opportunities: Automated Generation of Comprehensive, Personalized Programming Tasks](https://mlsanity.com/paper/2503.11704/) (2025)
- [LLM-Driven Feedback for Enhancing Conceptual Design Learning in Database Systems Courses](https://mlsanity.com/paper/2412.17892/) (2024)
- [The Role of Generative AI in Software Student CollaborAItion](https://mlsanity.com/paper/2501.14084/) (2025)
- [Open, Small, Rigmarole -- Evaluating Llama 3.2 3B's Feedback for Programming Exercises](https://mlsanity.com/paper/2504.01054/) (2025)
- [That's Not the Feedback I Need! -- Student Engagement with GenAI Feedback in the Tutor Kai](https://mlsanity.com/paper/2506.20433/) (2025)

## Built on by (most cited first, 1 of 1)

- [GenAI Voice Mode in Programming Education](https://mlsanity.com/paper/2509.10596/) (2025)

## Abstract (arXiv)

Building on prior research on Generative AI (GenAI) and related tools for programming education, we developed SCRIPT, a chatbot based on ChatGPT-4o-mini, to support novice learners. SCRIPT allows for open-ended interactions and structured guidance through predefined prompts. We evaluated the tool via an experiment with 136 students from an introductory programming course at a large German university and analyzed how students interacted with SCRIPT while solving programming tasks with a focus on their feedback preferences. The results reveal that students' feedback requests seem to follow a specific sequence. Moreover, the chatbot responses aligned well with students' requested feedback types (in 75%), and it adhered to the system prompt constraints. These insights inform the design of GenAI-based learning support systems and highlight challenges in balancing guidance and flexibility in AI-assisted tools.