# Reflective Dialogue or Prompt Refinement? Effects of Tutor Scaffolding on Students' Independent LLM Use for Programming

arXiv: https://arxiv.org/abs/2607.03303 · HTML: https://mlsanity.com/paper/2607.03303/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 35 references that are in the map.
- First posted on arXiv 3 Jul 2026.

## Builds on (most cited first, 11 of 11)

- [Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task](https://mlsanity.com/paper/2506.08872/) (2025)
- [Beware of Metacognitive Laziness: Effects of Generative Artificial Intelligence on Learning Motivation, Processes, and Performance](https://mlsanity.com/paper/2412.09315/) (2024)
- [Prompt Problems: A New Programming Exercise for the Generative AI Era](https://mlsanity.com/paper/2311.05943/) (2023)
- [Enhancing Critical Thinking in Education by means of a Socratic Chatbot](https://mlsanity.com/paper/2409.05511/) (2024)
- [Assigning AI: Seven Approaches for Students, with Prompts](https://mlsanity.com/paper/2306.10052/) (2023)
- [GPT-4 as a Homework Tutor can Improve Student Engagement and Learning Outcomes](https://mlsanity.com/paper/2409.15981/) (2024)
- [From Superficial Outputs to Superficial Learning: Risks of Large Language Models in Education](https://mlsanity.com/paper/2509.21972/) (2025)
- [Exploring Student Behaviors and Motivations using AI TAs with Optional Guardrails](https://mlsanity.com/paper/2504.11146/) (2025)
- [Sakshm AI: Advancing AI-Assisted Coding Education for Engineering Students in India Through Socratic Tutoring and Comprehensive Feedback](https://mlsanity.com/paper/2503.12479/) (2025)
- [Structured Prompts, Better Outcomes? Exploring the Effects of a Structured Interface with ChatGPT in a Graduate Robotics Course](https://mlsanity.com/paper/2507.07767/) (2025)
- [Exploring Conversational Design Choices in LLMs for Pedagogical Purposes: Socratic and Narrative Approaches for Improving Instructor's Teaching Practice](https://mlsanity.com/paper/2509.12107/) (2025)

## Abstract (arXiv)

While Large Language Models (LLMs) can provide personalized support in learning, several studies have raised concerns regarding their use in education. Importantly, learning depends on how students engage with LLMs. This study examined how two types of LLM-based tutors shape students' prompting practices, learning, and subsequent LLM-use: a Socratic-Guidance (SG) tutor, which structures interaction through dialogic questioning, and a Prompt-Refinement (PR) tutor that guides the formulation of effective prompts. We conducted a two-phase study in a graduate-level mobile robotics course: 66 students used either the SG or PR tutor during a 6-week intervention, followed by 52 students using an unconstrained LLM during a 3-week course project. Results show that while the SG- and PR tutors led to similar task performance and prompting patterns during guided use, they differ in learning outcomes and later LLM-use. SG-students, relative to PR-student, achieved higher learning gains in later sessions, and were more likely to adopt understanding-driven prompting strategies, which are predictive of higher understanding, when using an unconstrained LLM. Although learners perceived the SG tutor as less efficient, the findings suggest that Socratic guidance supports the development of students' capacity to learn with LLMs over time, highlighting its importance for LLM tutor design.