mlsanityData to 7 Oct 2026

ConvAI3: Generating Clarifying Questions for Open-Domain Dialogue Systems (ClariQ)

arXiv 2009.11352 ↗First posted 23 Sep 2020cs.CL cs.IRcited by 55

Not on the map. Why?

Abstract (arXiv)

This document presents a detailed description of the challenge on clarifying questions for dialogue systems (ClariQ). The challenge is organized as part of the Conversational AI challenge series (ConvAI3) at Search Oriented Conversational AI (SCAI) EMNLP workshop in 2020. The main aim of the conversational systems is to return an appropriate answer in response to the user requests. However, some user requests might be ambiguous. In IR settings such a situation is handled mainly thought the diversification of the search result page. It is however much more challenging in dialogue settings with limited bandwidth. Therefore, in this challenge, we provide a common evaluation framework to evaluate mixed-initiative conversations. Participants are asked to rank clarifying questions in an information-seeking conversations. The challenge is organized in two stages where in Stage 1 we evaluate the submissions in an offline setting and single-turn conversations. Top participants of Stage 1 get the chance to have their model tested by human annotators.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 9; 2023: 6; 2024: 10; 2025: 7; 2026: 14.

55 papers.

Sort:

By area:

  1. 2022CLAM: Selective Clarification for Ambiguous Questions with Generative Language Modelscited by 66
  2. 2021Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questionscited by 63
  3. 2022GODEL: Large-Scale Pre-Training for Goal-Directed Dialogcited by 50
  4. 2022Don't Blame the Annotator: Bias Already Starts in the Annotation Instructionscited by 47
  5. 2021Grounding 'Grounding' in NLPcited by 44
  6. 2022Neural Approaches to Conversational Information Retrievalcited by 41
  7. 2023Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedbackcited by 31
  8. 2022Evaluating Mixed-initiative Conversational Search Systems via User Simulationcited by 24
  9. 2023Asking Clarification Questions to Handle Ambiguity in Open-Domain QAcited by 24
  10. 2022Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021cited by 23
  11. 2024A Survey of Conversational Searchcited by 22
  12. 2023Post-Abstention: Towards Reliably Re-Attempting the Abstained Instances in QAcited by 20
38 more
  1. 2024STYLE: Improving Domain Transferability of Asking Clarification Questions in Large Language Model Powered Conversational Agentscited by 17
  2. 2026HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help?cited by 16
  3. 2021NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environmentcited by 15
  4. 2022IGLU 2022: Interactive Grounded Language Understanding in a Collaborative Environment at NeurIPS 2022cited by 14
  5. 2023Zero-shot Clarifying Question Generation for Conversational Searchcited by 14
  6. 2023Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructionscited by 13
  7. 2021Analysing Mixed Initiatives and Search Strategies during Conversational Searchcited by 12
  8. 2025ClarifyMT-Bench: Benchmarking and Improving Multi-Turn Clarification for Conversational Large Language Modelscited by 12
  9. 2021Context-Aware Target Apps Selection and Recommendation for Enhancing Personal Mobile Assistantscited by 11
  10. 2021Diverse and Specific Clarification Question Generation with Keywordscited by 10
  11. 2022Collecting Interactive Multi-modal Datasets for Grounded Language Understandingcited by 10
  12. 2024Asking Multimodal Clarifying Questions in Mixed-Initiative Conversational Searchcited by 8
  13. 2025AmbiK: Dataset of Ambiguous Tasks in Kitchen Environmentcited by 8
  14. 2024AGENT-CQ: Automatic Generation and Evaluation of Clarifying Questions for Conversational Search with LLMscited by 7
  15. 2024Learning to Ask: Conversational Product Search via Representation Learningcited by 7
  16. 2024CoSearchAgent: A Lightweight Collaborative Search Agent with Large Language Modelscited by 6
  17. 2025Disambiguation in Conversational Question Answering in the Era of LLMs and Agents: A Surveycited by 6
  18. 2021Conversational Search with Mixed-Initiative -- Asking Good Clarification Questions backed-up by Passage Retrievalcited by 5
  19. 2026LHAW: Controllable Underspecification for Long-Horizon Taskscited by 5
  20. 2025Multi-Turn Multi-Modal Question Clarification for Enhanced Conversational Understandingcited by 3
  21. 2025Curiosity by Design: An LLM-based Coding Assistant Asking Clarification Questionscited by 3
  22. 2024Corpus-informed Retrieval Augmented Generation of Clarifying Questionscited by 2
  23. 2025PublicAgent: Multi-Agent Design Principles From an LLM-Based Open Data Analysis Frameworkcited by 2
  24. 2026What Users Leave Unsaid: Under-Specified Queries Limit Vision-Language Modelscited by 2
  25. 2026Ask Early, Ask Late, Ask Right: When Does Clarification Timing Matter for Long-Horizon Agents?cited by 2
  26. 2021Keyword Extraction for Improved Document Retrieval in Conversational Searchcited by 1
  27. 2022Towards Building Economic Models of Conversational Searchcited by 1
  28. 2023Dialogue Agents 101: A Beginner's Guide to Critical Ingredients for Designing Effective Conversational Systemscited by 1
  29. 2024Uncertainty Resolution in Misinformation Detectioncited by 1
  30. 2024IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agentscited by 1
  31. 2025PSCon: Product Search Through Conversationscited by 1
  32. 2026ASPI: Seeking Ambiguity Clarification Amplifies Prompt Injection Vulnerability in LLM Agentscited by 1
  33. 2026Knowing but Not Showing: LLMs Recognize Ambiguity but Rarely Ask Clarifying Questionscited by 1
  34. 2021MICROS: Mixed-Initiative ConveRsatiOnal Systems Workshopcited by 0
  35. 2024SimpleStrat: Diversifying Language Model Generation with Stratificationcited by 0
  36. 2026Query Suggestion for Retrieval-Augmented Generation via Dynamic In-Context Learningcited by 0
  37. 2026Sensitivity-Aware Retrieval-Augmented Intent Clarificationcited by 0
  38. 2026Clarification Is Not Enough: Post-Clarification Answering Remains the Bottleneck in Multi-Turn QAcited by 0

50 most cited of 55 All 55 (JSON)