# Disfl-QA: A Benchmark Dataset for Understanding Disfluencies in Question Answering

arXiv: https://arxiv.org/abs/2106.04016 · HTML: https://mlsanity.com/paper/2106.04016/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 37 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [Generating Natural Adversarial Examples](https://mlsanity.com/paper/1710.11342/) (2017)
- [Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehension](https://mlsanity.com/paper/1804.00320/) (2018)
- [Did the Model Understand the Question?](https://mlsanity.com/paper/1805.05492/) (2018)
- [RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems](https://mlsanity.com/paper/2012.14666/) (2020)
- [Towards Data Distillation for End-to-end Spoken Conversational Question Answering](https://mlsanity.com/paper/2010.08923/) (2020)
- [Robustness Testing of Language Understanding in Task-Oriented Dialog](https://mlsanity.com/paper/2012.15262/) (2020)
- [Disfluency Detection using a Bidirectional LSTM](https://mlsanity.com/paper/1604.03209/) (2016)
- [Improving Disfluency Detection by Self-Training a Self-Attentive Model](https://mlsanity.com/paper/2004.05323/) (2020)
- [NoiseQA: Challenge Set Evaluation for User-Centric Question Answering](https://mlsanity.com/paper/2102.08345/) (2021)
- [Disfluencies and Human Speech Transcription Errors](https://mlsanity.com/paper/1904.04398/) (2019)
- [Multi-Task Self-Supervised Learning for Disfluency Detection](https://mlsanity.com/paper/1908.05378/) (2019)
- [Disfluency Detection using Auto-Correlational Neural Networks](https://mlsanity.com/paper/1808.09092/) (2018)
- [Disfluency Detection using a Noisy Channel Model and a Deep Neural Language Model](https://mlsanity.com/paper/1808.09091/) (2018)
- [Giving Attention to the Unexpected: Using Prosody Innovations in Disfluency Detection](https://mlsanity.com/paper/1904.04388/) (2019)
- [Mitigating Noisy Inputs for Question Answering](https://mlsanity.com/paper/1908.02914/) (2019)

## Built on by (most cited first, 20 of 20)

- [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://mlsanity.com/paper/2204.07705/) (2022)
- [QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension](https://mlsanity.com/paper/2107.12708/) (2021)
- [PRESTO: A Multilingual Dataset for Parsing Realistic Task-Oriented Dialogs](https://mlsanity.com/paper/2303.08954/) (2023)
- [Teaching BERT to Wait: Balancing Accuracy and Latency for Streaming Disfluency Detection](https://mlsanity.com/paper/2205.00620/) (2022)
- [LARD: Large-scale Artificial Disfluency Generation](https://mlsanity.com/paper/2201.05041/) (2022)
- [Streaming Joint Speech Recognition and Disfluency Detection](https://mlsanity.com/paper/2211.08726/) (2022)
- [A Survey on non-English Question Answering Dataset](https://mlsanity.com/paper/2112.13634/) (2021)
- [Answering Unanswered Questions through Semantic Reformulations in Spoken QA](https://mlsanity.com/paper/2305.17393/) (2023)
- [From Disfluency Detection to Intent Detection and Slot Filling](https://mlsanity.com/paper/2209.08359/) (2022)
- [MultiTurnCleanup: A Benchmark for Multi-Turn Spoken Conversational Transcript Cleanup](https://mlsanity.com/paper/2305.12029/) (2023)
- [Boosting Disfluency Detection with Large Language Model as Disfluency Generator](https://mlsanity.com/paper/2403.08229/) (2024)
- [Recording for Eyes, Not Echoing to Ears: Contextualized Spoken-to-Written Conversion of ASR Transcripts](https://mlsanity.com/paper/2408.09688/) (2024)
- [Artificial Disfluency Detection, Uh No, Disfluency Generation for the Masses](https://mlsanity.com/paper/2211.09235/) (2022)
- [DISCO: A Large Scale Human Annotated Corpus for Disfluency Correction in Indo-European Languages](https://mlsanity.com/paper/2310.16749/) (2023)
- [Efficiently Estimating Data Efficiency for Language Model Fine-tuning](https://mlsanity.com/paper/2512.24991/) (2025)
- [Mind the Pause: Disfluency-Aware Objective Tuning for Multilingual Speech Correction with LLMs](https://mlsanity.com/paper/2605.12242/) (2026)
- [DisfluencyFixer: A tool to enhance Language Learning through Speech To Speech Disfluency Correction](https://mlsanity.com/paper/2305.16957/) (2023)
- [5IDER: Unified Query Rewriting for Steering, Intent Carryover, Disfluencies, Entity Carryover and Repair](https://mlsanity.com/paper/2306.01855/) (2023)
- [Speakerly: A Voice-based Writing Assistant for Text Composition](https://mlsanity.com/paper/2310.16251/) (2023)
- [Should We Type or Talk to LLM Agents? A Comprehensive Study of Voice and Keyboard Input Perturbations](https://mlsanity.com/paper/2608.03970/) (2026)

## Abstract (arXiv)

Disfluencies is an under-studied topic in NLP, even though it is ubiquitous in human conversation. This is largely due to the lack of datasets containing disfluencies. In this paper, we present a new challenge question answering dataset, Disfl-QA, a derivative of SQuAD, where humans introduce contextual disfluencies in previously fluent questions. Disfl-QA contains a variety of challenging disfluencies that require a more comprehensive understanding of the text than what was necessary in prior datasets. Experiments show that the performance of existing state-of-the-art question answering models degrades significantly when tested on Disfl-QA in a zero-shot setting.We show data augmentation methods partially recover the loss in performance and also demonstrate the efficacy of using gold data for fine-tuning. We argue that we need large-scale disfluency datasets in order for NLP models to be robust to them. The dataset is publicly available at: https://github.com/google-research-datasets/disfl-qa.