mlsanityData to 14 Oct 2026

CLAM: Selective Clarification for Ambiguous Questions with Generative Language Models

arXiv 2212.07769 ↗First posted 15 Dec 2022cs.CL cs.AI cs.LGcited by 69

Not on the map. Why?

Abstract (arXiv)

Users often ask dialogue systems ambiguous questions that require clarification. We show that current language models rarely ask users to clarify ambiguous questions and instead provide incorrect answers. To address this, we introduce CLAM: a framework for getting language models to selectively ask for clarification about ambiguous user questions. In particular, we show that we can prompt language models to detect whether a given question is ambiguous, generate an appropriate clarifying question to ask the user, and give a final answer after receiving clarification. We also show that we can simulate users by providing language models with privileged information. This lets us automatically evaluate multi-turn clarification dialogues. Finally, CLAM significantly improves language models' accuracy on mixed ambiguous and unambiguous questions relative to SotA.

Key facts. Cited by 69 papers in the map (data through 14 Oct 2026). Builds on 14 of its 26 references that are in the map. First posted on arXiv 15 Dec 2022.

Builds on

14 of 26 references.

References · 14

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  4. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  6. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  7. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 390
  8. 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
  9. 2020ConvAI3: Generating Clarifying Questions for Open-Domain Dialogue Systems (ClariQ)cited by 58
  10. 2019Answer-based Adversarial Training for Generating Clarification Questionscited by 43
  11. 2020Resolving Intent Ambiguities by Retrieving Discriminative Clarifying Questionscited by 16
  12. 2016Continuously Learning Neural Dialogue Managementcited by 12
2 more
  1. 2019Identifying Unclear Questions in Community Question Answering Websitescited by 12
  2. 2022Conversational Agents: Theory and Applicationscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 17; 2025: 24; 2026: 26.

69 papers.

Sort:

By area:

  1. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 126
  2. 2024Language Model Cascades: Token-level uncertainty and beyondcited by 121
  3. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 114
  4. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 105
  5. 2024Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questionscited by 76
  6. 2025QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?cited by 63
  7. 2024Ask-before-Plan: Proactive Language Agents for Real-World Planningcited by 60
  8. 2024CLAMBER: A Benchmark of Identifying and Clarifying Ambiguous Information Needs in Large Language Modelscited by 54
  9. 2025What Prompts Don't Say: Understanding and Managing Underspecification in LLM Promptscited by 32
  10. 2024AmbigDocs: Reasoning across Documents on Different Entities under the Same Namecited by 21
  11. 2025From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centeredcited by 21
  12. 2025Semantic Volume: Quantifying and Detecting both External and Internal Uncertainty in LLMscited by 20
38 more
  1. 2024STYLE: Improving Domain Transferability of Asking Clarification Questions in Large Language Model Powered Conversational Agentscited by 17
  2. 2026Value of Information: A Framework for Human-Agent Communicationcited by 12
  3. 2024ProductAgent: Benchmarking Conversational Product Search Agent with Asking Clarification Questionscited by 10
  4. 2025InfoQuest: Evaluating Multi-Turn Dialogue Agents for Open-Ended Conversations with Hidden Contextcited by 8
  5. 2025CaRT: Teaching LLM Agents to Know When They Know Enoughcited by 7
  6. 2025Disambiguation in Conversational Question Answering in the Era of LLMs and Agents: A Surveycited by 6
  7. 2025Referential ambiguity and clarification requests: comparing human and LLM behaviourcited by 5
  8. 2025Beyond Passive Critical Thinking: Fostering Proactive Questioning to Enhance Human-AI Collaborationcited by 5
  9. 2025Sparse Neurons Carry Strong Signals of Question Ambiguity in LLMscited by 5
  10. 2025Learning Steerable Clarification Policies with Collaborative Self-playcited by 5
  11. 2026Ambig-DS: A Benchmark for Task-Framing Ambiguity in Data-Science Agentscited by 5
  12. 2024Predict the Next Word: Humans exhibit uncertainty in this task and language models _____cited by 4
  13. 2025Zero-Shot and Efficient Clarification Need Prediction in Conversational Searchcited by 4
  14. 2026Ask Early, Ask Late, Ask Right: When Does Clarification Timing Matter for Long-Horizon Agents?cited by 4
  15. 2024Engineering Conversational Search Systems: A Review of Applications, Architectures, and Functional Componentscited by 3
  16. 2025It Depends: Resolving Referential Ambiguity in Minimal Contexts with Commonsense Knowledgecited by 3
  17. 2026VLM-UQBench: A Benchmark for Modality-Specific and Cross-Modality Uncertainties in Vision Language Modelscited by 3
  18. 2026Knowing but Not Showing: LLMs Recognize Ambiguity but Rarely Ask Clarifying Questionscited by 3
  19. 2024Uncertainty Resolution in Misinformation Detectioncited by 2
  20. 2024Variability Need Not Imply Error: The Case of Adequate but Semantically Distinct Responsescited by 2
  21. 2025From Misleading Queries to Accurate Answers: A Three-Stage Fine-Tuning Method for LLMscited by 2
  22. 2025Agents Require Metacognitive and Strategic Reasoning to Succeed in the Coming Labor Marketscited by 2
  23. 2025ChemAU: Harness the Reasoning of LLMs in Chemical Research with Adaptive Uncertainty Estimationcited by 2
  24. 2025First Ask Then Answer: A Framework Design for AI Dialogue Based on Supplementary Questioning with Large Language Modelscited by 2
  25. 2025Can Multiple Responses from an LLM Reveal the Sources of Its Uncertainty?cited by 2
  26. 2026Pause or Fabricate? Training Language Models for Grounded Reasoningcited by 2
  27. 2026SCICONVBENCH: Benchmarking LLMs on Multi-Turn Clarification for Task Formulation in Computational Sciencecited by 2
  28. 2024Generating Clarification Questions for Disambiguating Contractscited by 1
  29. 2024It Couldn't Help But Overhear: On the Limits of Modelling Meta-Communicative Grounding Acts with Supervised Learningcited by 1
  30. 2025Beyond Solving Math Quiz: Evaluating the Ability of Large Reasoning Models to Ask for Informationcited by 1
  31. 2025Identifying & Interactively Refining Ambiguous User Goals for Data Visualization Code Generationcited by 1
  32. 2025NRR-Core: Non-Resolution Reasoning as a Computational Framework for Contextual Identity and Ambiguity Preservationcited by 1
  33. 2026Do Reasoning Models Ask Better Questions? A Formal Information-Theoretic Analysis on Multi-Turn LLM Gamescited by 1
  34. 2026Game of Thought: Robust Information Seeking with Large Language Models Using Game Theorycited by 1
  35. 2026Spilled Energy in Large Language Modelscited by 1
  36. 2026LLM-as-an-Investigator: Evidence-First Reasoning for Robust Interactive Problem Diagnosiscited by 1
  37. 2024Tool-Augmented LLMs as a Universal Interface for IDEscited by 0
  38. 2024Follow-Up Questions Improve Documents Generated by Large Language Modelscited by 0

50 most cited of 69 All 69 (JSON)