# HeySQuAD: A Spoken Question Answering Dataset

arXiv: https://arxiv.org/abs/2304.13689 · HTML: https://mlsanity.com/paper/2304.13689/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 28 references that are in the map.
- First posted on arXiv 26 Apr 2023.

## Builds on (most cited first, 18 of 18)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [fairseq S2T: Fast Speech-to-Text Modeling with fairseq](https://mlsanity.com/paper/2010.05171/) (2020)
- [Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehension](https://mlsanity.com/paper/1804.00320/) (2018)
- [Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answering](https://mlsanity.com/paper/2109.03381/) (2021)
- [SD-QA: Spoken Dialectal Question Answering for the Real World](https://mlsanity.com/paper/2109.12072/) (2021)
- [Towards Data Distillation for End-to-end Spoken Conversational Question Answering](https://mlsanity.com/paper/2010.08923/) (2020)
- [SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks](https://mlsanity.com/paper/2212.10525/) (2022)
- [Knowledge Distillation for Improved Accuracy in Spoken Question Answering](https://mlsanity.com/paper/2010.11067/) (2020)
- [Self-supervised Dialogue Learning for Spoken Conversational Question Answering](https://mlsanity.com/paper/2106.02182/) (2021)
- [Contextualized Attention-based Knowledge Transfer for Spoken Conversational Question Answering](https://mlsanity.com/paper/2010.11066/) (2020)
- [Towards Machine Comprehension of Spoken Content: Initial TOEFL Listening Comprehension Test by Machine](https://mlsanity.com/paper/1608.06378/) (2016)
- [ODSQA: Open-domain Spoken Question Answering Dataset](https://mlsanity.com/paper/1808.02280/) (2018)
- [SpeechBERT: An Audio-and-text Jointly Learned Language Model for End-to-end Spoken Question Answering](https://mlsanity.com/paper/1910.11559/) (2019)
- [Mitigating the Impact of Speech Recognition Errors on Spoken Question Answering by Adversarial Domain Adaptation](https://mlsanity.com/paper/1904.07904/) (2019)

## Built on by (most cited first, 16 of 16)

- [SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words](https://mlsanity.com/paper/2406.13340/) (2024)
- [Distilling an End-to-End Voice Assistant Without Instruction Training Data](https://mlsanity.com/paper/2410.02678/) (2024)
- [Mind the Gap! Static and Interactive Evaluations of Large Audio Models](https://mlsanity.com/paper/2502.15919/) (2025)
- [SpokenNativQA: Multilingual Everyday Spoken Queries for LLMs](https://mlsanity.com/paper/2505.19163/) (2025)
- [WearVox: An Egocentric Multichannel Voice Assistant Benchmark for Wearables](https://mlsanity.com/paper/2601.02391/) (2025)
- [The Role of Prosody in Spoken Question Answering](https://mlsanity.com/paper/2502.05389/) (2025)
- [Spoken question answering for visual queries](https://mlsanity.com/paper/2505.23308/) (2025)
- [ALARM: Audio-Language Alignment for Reasoning Models](https://mlsanity.com/paper/2603.09556/) (2026)
- [AuRA: Internalizing Audio Understanding into LLMs as LoRA](https://mlsanity.com/paper/2606.11033/) (2026)
- [Synera: Synergistic LLM Serving across Device and Cloud at Scale](https://mlsanity.com/paper/2511.07423/) (2025)
- [ATIR: Towards Audio-Text Interleaved Contextual Retrieval](https://mlsanity.com/paper/2604.20267/) (2026)
- [KVoiceBench, KOpenAudioBench, and KMMAU: Agent-Driven Korean Speech Benchmarks for Evaluating SpeechLMs](https://mlsanity.com/paper/2605.27984/) (2026)
- [LuxSQA: Ask Me in Luxembourgish with TTS-Augmented Spoken Question Answering](https://mlsanity.com/paper/2607.02763/) (2026)
- [Don't Just Listen, Try Planning: Graph-based Retrieval-Generation Agent for Long-form Audio Meeting Understanding](https://mlsanity.com/paper/2608.24048/) (2026)
- [The Bairong System for MLC-SLM 2026: Dynamic Question-Aware Evidence Routing for Multilingual Conversational Speech Understanding](https://mlsanity.com/paper/2609.22214/) (2026)
- [Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation](https://mlsanity.com/paper/2610.02148/) (2026)

## Abstract (arXiv)

Spoken question answering (SQA) systems are critical for digital assistants and other real-world use cases, but evaluating their performance is a challenge due to the importance of human-spoken questions. This study presents a new large-scale community-shared SQA dataset called HeySQuAD, which includes 76k human-spoken questions, 97k machine-generated questions, and their corresponding textual answers from the SQuAD QA dataset. Our goal is to measure the ability of machines to accurately understand noisy spoken questions and provide reliable answers. Through extensive testing, we demonstrate that training with transcribed human-spoken and original SQuAD questions leads to a significant improvement (12.51%) in answering human-spoken questions compared to training with only the original SQuAD textual questions. Moreover, evaluating with a higher-quality transcription can lead to a further improvement of 2.03%. This research has significant implications for the development of SQA systems and their ability to meet the needs of users in real-world scenarios.