# Curiosity by Design: An LLM-based Coding Assistant Asking Clarification Questions

arXiv: https://arxiv.org/abs/2507.21285 · HTML: https://mlsanity.com/paper/2507.21285/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 48 references that are in the map.
- First posted on arXiv 28 Jul 2025.

## Builds on (most cited first, 30 of 32)

- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Grounded Copilot: How Programmers Interact with Code-Generating Models](https://mlsanity.com/paper/2206.15000/) (2022)
- [STaR-GATE: Teaching Language Models to Ask Clarifying Questions](https://mlsanity.com/paper/2403.19154/) (2024)
- [The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Development](https://mlsanity.com/paper/2302.07080/) (2023)
- [LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigation](https://mlsanity.com/paper/2409.20550/) (2024)
- [Towards an Understanding of Large Language Models in Software Engineering Tasks](https://mlsanity.com/paper/2308.11396/) (2023)
- [A Survey on Large Language Models for Software Engineering](https://mlsanity.com/paper/2312.15223/) (2023)
- [ConvAI3: Generating Clarifying Questions for Open-Domain Dialogue Systems (ClariQ)](https://mlsanity.com/paper/2009.11352/) (2020)
- [Can Large Language Models Transform Natural Language Intent into Formal Method Postconditions?](https://mlsanity.com/paper/2310.01831/) (2023)
- [How Novices Use LLM-Based Code Generators to Solve CS1 Coding Tasks in a Self-Paced Learning Environment](https://mlsanity.com/paper/2309.14049/) (2023)
- [LLM-Based Test-Driven Interactive Code Generation: User Study and Empirical Evaluation](https://mlsanity.com/paper/2404.10100/) (2024)
- [Using AI-Based Coding Assistants in Practice: State of Affairs, Perceptions, and Ways Forward](https://mlsanity.com/paper/2406.07765/) (2024)
- [Need Help? Designing Proactive AI Assistants for Programming](https://mlsanity.com/paper/2410.04596/) (2024)
- [Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions](https://mlsanity.com/paper/2308.02312/) (2023)
- [Test-Driven Development for Code Generation](https://mlsanity.com/paper/2402.13521/) (2024)
- [Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review](https://mlsanity.com/paper/2307.02503/) (2023)
- [ClarQ: A large-scale and diverse dataset for Clarification Question Generation](https://mlsanity.com/paper/2006.05986/) (2020)
- [How Beginning Programmers and Code LLMs (Mis)read Each Other](https://mlsanity.com/paper/2401.15232/) (2024)
- [A Survey on Asking Clarification Questions Datasets in Conversational Systems](https://mlsanity.com/paper/2305.15933/) (2023)
- [Beyond Code Generation: An Observational Study of ChatGPT Usage in Software Engineering Practice](https://mlsanity.com/paper/2404.14901/) (2024)
- [How Do Data Analysts Respond to AI Assistance? A Wizard-of-Oz Study](https://mlsanity.com/paper/2309.10108/) (2023)
- [ClarQ-LLM: A Benchmark for Models Clarifying and Requesting Information in Task-Oriented Dialog](https://mlsanity.com/paper/2409.06097/) (2024)
- [LLMs in Web Development: Evaluating LLM-Generated PHP Code Unveiling Vulnerabilities and Limitations](https://mlsanity.com/paper/2404.14459/) (2024)
- [A Pair Programming Framework for Code Generation via Multi-Plan Exploration and Feedback-Driven Refinement](https://mlsanity.com/paper/2409.05001/) (2024)
- [Can We Trust Large Language Models Generated Code? A Framework for In-Context Learning, Security Patterns, and Code Evaluations Across Diverse LLMs](https://mlsanity.com/paper/2406.12513/) (2024)
- [Challenges in Trustworthy Human Evaluation of Chatbots](https://mlsanity.com/paper/2412.04363/) (2024)
- [Can ChatGPT Support Developers? An Empirical Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2402.11702/) (2024)
- [On the Effectiveness of Creating Conversational Agent Personalities Through Prompting](https://mlsanity.com/paper/2310.11182/) (2023)
- [Understanding the Human-LLM Dynamic: A Literature Survey of LLM Use in Programming Tasks](https://mlsanity.com/paper/2410.01026/) (2024)
- [In-IDE Human-AI Experience in the Era of Large Language Models; A Literature Review](https://mlsanity.com/paper/2401.10739/) (2024)

## Built on by (most cited first, 4 of 4)

- [Ask or Assume? Uncertainty-Aware Clarification-Seeking in Coding Agents](https://mlsanity.com/paper/2603.26233/) (2026)
- [Asking What Matters: Reward-Driven Clarification for Software Engineering Tasks](https://mlsanity.com/paper/2604.14624/) (2026)
- [Closing the Expression Gap in LLM Instructions via Socratic Questioning](https://mlsanity.com/paper/2510.27410/) (2025)
- [Pareto-Dominant Clarification: Post-Training Coding LLMs via PPO-Lagrangian Budget Constraints](https://mlsanity.com/paper/2610.04089/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly used as coding assistants. However, the ambiguity of the developer's prompt often leads to incorrect code generation, as current models struggle to infer user intent without extensive prompt engineering or external context. This work aims to build an LLM-based coding assistant that mimics the human code review process by asking clarification questions when faced with ambiguous or under-specified queries. Our end-to-end system includes (1) a query classifier trained to detect unclear programming-related queries and (2) a fine-tuned LLM that generates clarification questions. Our evaluation shows that the fine-tuned LLM outperforms standard zero-shot prompting in generating useful clarification questions. Furthermore, our user study indicates that users find the clarification questions generated by our model to outperform the baseline, demonstrating that our coding assistant produces more accurate and helpful code responses compared to baseline coding assistants.