mlsanityData to 7 Oct 2026

STaR-GATE: Teaching Language Models to Ask Clarifying Questions

arXiv 2403.19154 ↗First posted 28 Mar 2024cs.CL cs.AIcited by 99

Not on the map. Why?

Abstract (arXiv)

When prompting language models to complete a task, users often leave important aspects unsaid. While asking questions could resolve this ambiguity (GATE; Li et al., 2023), models often struggle to ask good questions. We explore a language model's ability to self-improve (STaR; Zelikman et al., 2022) by rewarding the model for generating useful questions-a simple method we dub STaR-GATE. We generate a synthetic dataset of 25,500 unique persona-task prompts to simulate conversations between a pretrained language model-the Questioner-and a Roleplayer whose preferences are unknown to the Questioner. By asking questions, the Questioner elicits preferences from the Roleplayer. The Questioner is iteratively finetuned on questions that increase the probability of high-quality responses to the task, which are generated by an Oracle with access to the Roleplayer's latent preferences. After two iterations of self-improvement, the Questioner asks better questions, allowing it to generate responses that are preferred over responses from the initial model on 72% of tasks. Our results indicate that teaching a language model to ask better questions leads to better personalized responses.

Key facts. Cited by 99 papers in the map (data through 7 Oct 2026). Builds on 35 of its 46 references that are in the map. First posted on arXiv 28 Mar 2024.

Builds on

35 of 46 references.

References · 35

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  7. 2017Deep reinforcement learning from human preferencescited by 3,846
  8. 2023Mistral 7Bcited by 3,065
  9. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  10. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  11. 2024Mixtral of Expertscited by 1,785
  12. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
23 more
  1. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  2. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  3. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
  4. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  5. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  6. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  7. 2018Probabilistic Model-Agnostic Meta-Learningcited by 254
  8. 2017Thinking Fast and Slow with Deep Learning and Tree Searchcited by 245
  9. 2024V-STaR: Training Verifiers for Self-Taught Reasonerscited by 214
  10. 2018Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Informationcited by 110
  11. 2023Eliciting Human Preferences with Language Modelscited by 90
  12. 2023Strategic Reasoning with Language Modelscited by 64
  13. 2021Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questionscited by 63
  14. 2017Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generationcited by 59
  15. 2023Training Chain-of-Thought via Latent-Variable Inferencecited by 46
  16. 2019Answer-based Adversarial Training for Generating Clarification Questionscited by 42
  17. 2024In-Context Principle Learning from Mistakescited by 42
  18. 2023Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversationscited by 38
  19. 2024Bayesian Preference Elicitation with Language Modelscited by 32
  20. 2023Decision-Oriented Dialogue for Human-AI Collaborationcited by 29
  21. 2023Active Preference Inference using Language Models and Probabilistic Reasoningcited by 29
  22. 2022Task Ambiguity in Humans and Language Modelscited by 27
  23. 2023Social Contract AI: Aligning AI Assistants with Implicit Group Normscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 19; 2025: 34; 2026: 46.

99 papers.

Sort:

By area:

  1. 2025CollabLLM: From Passive Responders to Active Collaboratorscited by 117
  2. 2024Can LLM be a Personalized Judge?cited by 99
  3. 2024PERSONA: A Reproducible Testbed for Pluralistic Alignmentcited by 71
  4. 2024Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questionscited by 70
  5. 2025A Survey on (M)LLM-Based GUI Agentscited by 61
  6. 2024Ask-before-Plan: Proactive Language Agents for Real-World Planningcited by 59
  7. 2025QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?cited by 56
  8. 2025Creating General User Models from Computer Usecited by 44
  9. 2025PersonaLens: A Benchmark for Personalization Evaluation in Conversational AI Assistantscited by 44
  10. 2024Learning to Ask: When LLM Agents Meet Unclear Instructioncited by 38
  11. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  12. 2025Bayesian Teaching Enables Probabilistic Reasoning in Large Language Modelscited by 36
38 more
  1. 2024Aligning Language Models with Demonstrated Feedbackcited by 32
  2. 2025Navigating Rifts in Human-LLM Grounding: Study and Benchmarkcited by 27
  3. 2025ALFA: Aligning LLMs to Ask Good Questions A Case Study in Clinical Reasoningcited by 24
  4. 2024A Survey of Conversational Searchcited by 22
  5. 2025BED-LLM: Intelligent Information Gathering with LLMs and Bayesian Experimental Designcited by 21
  6. 2025CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactionscited by 20
  7. 2024Self-Supervised Alignment with Mutual Information: Learning to Follow Principles without Preference Labelscited by 18
  8. 2024Talk Less, Interact Better: Evaluating In-context Conversational Adaptation in Multimodal LLMscited by 18
  9. 2026Ask or Assume? Uncertainty-Aware Clarification-Seeking in Coding Agentscited by 15
  10. 2024ReSpAct: Harmonizing Reasoning, Speaking, and Acting Towards Building Large Language Model-Based Conversational AI Agentscited by 14
  11. 2024Social Skill Training with Large Language Modelscited by 13
  12. 2024Dialogue Action Tokens: Steering Language Models in Goal-Directed Dialogue with a Multi-Turn Plannercited by 12
  13. 2024Contextualized Evaluations: Judging Language Model Responses to Underspecified Queriescited by 12
  14. 2025AskToAct: Enhancing LLMs Tool Use via Self-Correcting Clarificationcited by 12
  15. 2025Persona-judge: Personalized Alignment of Large Language Models via Token-level Self-judgmentcited by 11
  16. 2025Offline RL by Reward-Weighted Fine-Tuning for Conversation Optimizationcited by 10
  17. 2026ConvApparel: A Benchmark Dataset and Validation Framework for User Simulators in Conversational Recommenderscited by 10
  18. 2025A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directionscited by 9
  19. 2025Charts Are Not Images: On the Challenges of Scientific Chart Editingcited by 9
  20. 2026Learning to Simulate Human Dialoguecited by 9
  21. 2025Moloch's Bargain: Emergent Misalignment When LLMs Compete for Audiencescited by 7
  22. 2026MultiSessionCollab: Learning User Preferences with Memory to Improve Long-Term Collaborationcited by 7
  23. 2024MAPLE: A Framework for Active Preference Learning Guided by Large Language Modelscited by 6
  24. 2024AI and the Future of Digital Public Squarescited by 6
  25. 2025Towards Evaluating Proactive Risk Awareness of Multimodal Language Modelscited by 6
  26. 2025Collaborate, Deliberate, Evaluate: How LLM Alignment Affects Coordinated Multi-Agent Outcomescited by 6
  27. 2025Asking Clarifying Questions for Preference Elicitation With Large Language Modelscited by 6
  28. 2026Quantifying the Utility of User Simulators for Building Collaborative LLM Assistantscited by 6
  29. 2025ADAPT: Actively Discovering and Adapting to Preferences for any Taskcited by 5
  30. 2025TO-GATE: Clarifying Questions and Summarizing Responses with Trajectory Optimization for Eliciting Human Preferencecited by 5
  31. 2025The Curious Language Model: Strategic Test-Time Information Acquisitioncited by 5
  32. 2025Referential ambiguity and clarification requests: comparing human and LLM behaviourcited by 5
  33. 2025Beyond Passive Critical Thinking: Fostering Proactive Questioning to Enhance Human-AI Collaborationcited by 5
  34. 2024Procedural Dilemma Generation for Evaluating Moral Reasoning in Humans and Language Modelscited by 4
  35. 2024RL-STaR: Theoretical Analysis of Reinforcement Learning Frameworks for Self-Taught Reasonercited by 4
  36. 2025Query Understanding in LLM-based Conversational Information Seekingcited by 4
  37. 2026Measuring and Mitigating the Distributional Gap Between Real and Simulated User Behaviorscited by 4
  38. 2026ProactBench: Beyond What The User Asked Forcited by 4

50 most cited of 99 All 99 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves
  2. 2025Self-Questioning Language Models
  3. 2023Self-Convinced Prompting: Few-Shot Question Answering with Repeated Introspection
  4. 2025SpeakRL: Synergizing Reasoning, Speaking, and Acting in Language Models with Reinforcement Learning
  5. 2022Language Models (Mostly) Know What They Know
  6. 2024Learning to Ask Informative Questions: Enhancing LLMs with Preference Optimization and Expected Information Gain
  7. 2024Rewriting Conversational Utterances with Instructed Large Language Models
  8. 2026What Am I Missing? Question-Answering as Hidden State Probing