mlsanityData to 7 Oct 2026

CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answering

arXiv 2502.01523 ↗First posted 3 Feb 2025cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

Users often assume that large language models (LLMs) share their cognitive alignment of context and intent, leading them to omit critical information in question-answering (QA) and produce ambiguous queries. Responses based on misaligned assumptions may be perceived as hallucinations. Therefore, identifying possible implicit assumptions is crucial in QA. To address this fundamental challenge, we propose Conditional Ambiguous Question-Answering (CondAmbigQA), a benchmark comprising 2,000 ambiguous queries and condition-aware evaluation metrics. Our study pioneers "conditions" as explicit contextual constraints that resolve ambiguities in QA tasks through retrieval-based annotation, where retrieved Wikipedia fragments help identify possible interpretations for a given query and annotate answers accordingly. Experiments demonstrate that models considering conditions before answering improve answer accuracy by 11.75%, with an additional 7.15% gain when conditions are explicitly provided. These results highlight that apparent hallucinations may stem from inherent query ambiguity rather than model failure, and demonstrate the effectiveness of condition reasoning in QA, providing researchers with tools for rigorous evaluation.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 53 of its 70 references that are in the map. First posted on arXiv 3 Feb 2025.

Builds on

53 of 70 references.

References · 53

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  9. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  10. 2023Mistral 7Bcited by 3,065
  11. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  12. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
38 more
  1. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  2. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  4. 2024Qwen2 Technical Reportcited by 2,076
  5. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  7. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  8. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  9. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  10. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  11. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  12. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  13. 2019ELI5: Long Form Question Answeringcited by 567
  14. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
  15. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  16. 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 302
  17. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
  18. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  19. 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queriescited by 253
  20. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  21. 2024LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Taskscited by 236
  22. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
  23. 2024Corrective Retrieval Augmented Generationcited by 187
  24. 2019Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasetscited by 173
  25. 2024Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Toolscited by 170
  26. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  27. 2024LLMs Will Always Hallucinate, and We Need to Live With Thiscited by 77
  28. 2022Solving Math Word Problems via Cooperative Reasoning induced Language Modelscited by 76
  29. 2024RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generationcited by 75
  30. 2024Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluationcited by 73
  31. 2024MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixturescited by 73
  32. 2024Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agentscited by 71
  33. 2024Aligning Language Models to Explicitly Handle Ambiguitycited by 55
  34. 2023DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacycited by 34
  35. 2024Evaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructionscited by 34
  36. 2024CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responsescited by 23
  37. 2024Open Domain Question Answering with Conflicting Contextscited by 18
  38. 2024Adaptive Question Answering: Enhancing Language Model Proficiency for Addressing Knowledge Conflicts with Source Citationscited by 15

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 12.

15 papers.

Sort:

By area:

  1. 2025BIRD-INTERACT: Re-imagining Text-to-SQL Evaluation for Large Language Models via Lens of Dynamic Interactionscited by 17
  2. 2026Drift-Bench: Diagnosing Cooperative Breakdowns in LLM Agents under Input Faults via Multi-Turn Interactioncited by 7
  3. 2025DEEPAMBIGQA: Ambiguous Multi-hop Questions for Benchmarking LLM Answer Completenesscited by 6
  4. 2026LLMs Exhibit Significantly Lower Uncertainty in Creative Writing Than Professional Writerscited by 4
  5. 2026BMAM: Brain-inspired Multi-Agent Memory Frameworkcited by 3
  6. 2025MARCH: Evaluating the Intersection of Ambiguity Interpretation and Multi-hop Inferencecited by 1
  7. 2026AQuA: Toward Strategic Response Generation for Ambiguous Visual Questionscited by 1
  8. 2026AUDITA: A New Dataset to Audit Humans vs. AI Skill at Audio QAcited by 1
  9. 2026SCICONVBENCH: Benchmarking LLMs on Multi-Turn Clarification for Task Formulation in Computational Sciencecited by 1
  10. 2026When Search Agents Should Ask: DiscoBench for Clarification-Aware Deep Searchcited by 1
  11. 2026Clarify, Abstain or Answer? Strategising in Conversation with Belief-Augmented Generationcited by 0
  12. 2026AURA: Intent-Directed Probing for Implicit-Need Surfacing in Situated LLM Agentscited by 0
3 more
  1. 2026One More Turn, Less Regret: A Regret-Based Multi-Turn Benchmark for LLMs' Clarification Policiescited by 0
  2. 2026Policy Loopholes in Agent Evaluation: When Policy Ambiguity Masquerades as Agent Errorcited by 0
  3. 2026FinInteract: Benchmarking Clarification and Intent Integration in Ambiguous Financial Question Answeringcited by 0