# Deciding Whether to Ask Clarifying Questions in Large-Scale Spoken Language Understanding

arXiv: https://arxiv.org/abs/2109.12451 · HTML: https://mlsanity.com/paper/2109.12451/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 25 Sep 2021.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [AmbigQA: Answering Ambiguous Open-domain Questions](https://mlsanity.com/paper/2004.10645/) (2020)
- [Asking Clarifying Questions in Open-Domain Information-Seeking Conversations](https://mlsanity.com/paper/1907.06554/) (2019)
- [Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks](https://mlsanity.com/paper/1810.00825/) (2018)
- [Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information](https://mlsanity.com/paper/1805.04655/) (2018)
- [Learning from Dialogue after Deployment: Feed Yourself, Chatbot!](https://mlsanity.com/paper/1901.05415/) (2019)
- [ClarQ: A large-scale and diverse dataset for Clarification Question Generation](https://mlsanity.com/paper/2006.05986/) (2020)
- [MIMICS: A Large-Scale Data Collection for Search Clarification](https://mlsanity.com/paper/2006.10174/) (2020)
- [Is Your Goal-Oriented Dialog Model Performing Really Well? Empirical Analysis of System-wise Evaluation](https://mlsanity.com/paper/2005.07362/) (2020)
- [Dialog Policy Learning for Joint Clarification and Active Learning Queries](https://mlsanity.com/paper/2006.05456/) (2020)
- [Just ASK: Building an Architecture for Extensible Self-Service Spoken Language Understanding](https://mlsanity.com/paper/1711.00549/) (2017)
- [A Scalable Neural Shortlisting-Reranking Approach for Large-Scale Domain Classification in Natural Language Understanding](https://mlsanity.com/paper/1804.08064/) (2018)
- [Pseudo Labeling and Negative Feedback Learning for Large-scale Multi-label Domain Classification](https://mlsanity.com/paper/2003.03728/) (2020)
- [Efficient Large-Scale Domain Classification with Personalized Attention](https://mlsanity.com/paper/1804.08065/) (2018)
- [Large-scale Hybrid Approach for Predicting User Satisfaction with Conversational Agents](https://mlsanity.com/paper/2006.07113/) (2020)
- [Supervised Domain Enablement Attention for Personalized Domain Classification](https://mlsanity.com/paper/1812.07546/) (2018)

## Built on by (most cited first, 5 of 5)

- [Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedback](https://mlsanity.com/paper/2304.10750/) (2023)
- [Zero-Shot and Efficient Clarification Need Prediction in Conversational Search](https://mlsanity.com/paper/2503.00179/) (2025)
- [Discourse over Discourse: The Need for an Expanded Pragmatic Focus in Conversational AI](https://mlsanity.com/paper/2304.14543/) (2023)
- [Expanding the Set of Pragmatic Considerations in Conversational AI](https://mlsanity.com/paper/2310.18435/) (2023)
- [Clarification Is Not Enough: Post-Clarification Answering Remains the Bottleneck in Multi-Turn QA](https://mlsanity.com/paper/2605.25204/) (2026)

## Abstract (arXiv)

A large-scale conversational agent can suffer from understanding user utterances with various ambiguities such as ASR ambiguity, intent ambiguity, and hypothesis ambiguity. When ambiguities are detected, the agent should engage in a clarifying dialog to resolve the ambiguities before committing to actions. However, asking clarifying questions for all the ambiguity occurrences could lead to asking too many questions, essentially hampering the user experience. To trigger clarifying questions only when necessary for the user satisfaction, we propose a neural self-attentive model that leverages the hypotheses with ambiguities and contextual signals. We conduct extensive experiments on five common ambiguity types using real data from a large-scale commercial conversational agent and demonstrate significant improvement over a set of baseline approaches.