Prompt Coach: An Empirical Evaluation of an Agentic Tutor for Learning Prompt Engineering in Software Development
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Prompt engineering has emerged as a critical yet undertaught skill for software developers, one that traditional learning approaches are ill-equipped to support given its evolving, interactive, and context-dependent nature. In this paper, we introduce Prompt Coach (PC), an agentic tutor that helps developers learn how to craft high-quality code-generation prompts through Socratic guidance embedded in-flow within their IDE. PC evaluates prompt quality across multiple dimensions and surfaces targeted questions to guide self-correction, grounded in the developer's codebase and the behavior of the target LLM. We present an early empirical study with 15 professional developers combining quantitative prompt quality scoring with qualitative perception measures. Participants showed statistically significant improvements after a single 60-minute session, with the largest gains across dimensions commonly overlooked by developers. They also reported strong trust, high adoption readiness, and unanimous agreement that PC improved their prompt-writing skills.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 14 of its 35 references that are in the map. First posted on arXiv 7 Jul 2026.
Builds on
14 of 35 references.
References · 14
- 2021Measuring Coding Challenge Competence With APPScited by 879
- 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
- 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 216
- 2023ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testingcited by 88
- 2024Prompting Techniques for Secure Code Generation: A Systematic Investigationcited by 33
- 2024Test-Driven Development for Code Generationcited by 32
- 2025One Size doesn't Fit All: A Personalized Conversational Tutoring Agent for Mathematics Instructioncited by 17
- 2023From Misuse to Mastery: Enhancing Code Generation with Knowledge-Driven AI Chainingcited by 15
- 2024Instruct, Not Assist: LLM-based Multi-Turn Planning and Hierarchical Questioning for Socratic Code Debuggingcited by 11
- 2025Prompt Engineer: Analyzing Hard and Soft Skill Requirements in the AI Job Marketcited by 3
- 2026Guidelines to Prompt Large Language Models for Code Generation: An Empirical Characterizationcited by 3
- 2025What do professional software developers need to know to succeed in an age of Artificial Intelligence?cited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026REprompt: Prompt Generation for Intelligent Software Development Guided by Requirements Engineering ↗
- 2026A Tutorial on Prompt Engineering: From Messy Thoughts to AI Workflows
- 2026Do Personalized Skills Help Coding Agents? An Empirical Study of Developer Interaction Histories
- 2026AEVAL: From Anecdotal to Deterministic Testing for Agentic Skill Workflows
- 2026SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?
- 2026Rethinking the Value of Agent-Generated Tests for LLM-Based Software Engineering Agents
- 2026Prompt-Driven Code Summarization: A Systematic Literature Review
- 2025Representing Prompting Patterns with PDL: Compliance Agent Case Study