Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehension
Not on the map. Why?
Abstract (arXiv)
Reading comprehension has been widely studied. One of the most representative reading comprehension tasks is Stanford Question Answering Dataset (SQuAD), on which machine is already comparable with human. On the other hand, accessing large collections of multimedia or spoken content is much more difficult and time-consuming than plain text content for humans. It's therefore highly attractive to develop machines which can automatically understand spoken content. In this paper, we propose a new listening comprehension task - Spoken SQuAD. On the new task, we found that speech recognition errors have catastrophic impact on machine comprehension, and several approaches are proposed to mitigate the impact.
Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Apr 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 9; 2022: 7; 2023: 6; 2024: 15; 2025: 16; 2026: 21.
74 papers.
Sort:
By area:
- 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
- 2024SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Wordscited by 67
- 2025Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generationcited by 42
- 2021Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answeringcited by 39
- 2022End-to-end Spoken Conversational Question Answering: Task, Dataset and Modelcited by 37
- 2021SD-QA: Spoken Dialectal Question Answering for the Real Worldcited by 36
- 2022SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Taskscited by 34
- 2021Self-supervised Dialogue Learning for Spoken Conversational Question Answeringcited by 27
- 2021M5Product: Self-harmonized Contrastive Learning for E-commercial Multi-modal Pretrainingcited by 26
- 2021NoiseQA: Challenge Set Evaluation for User-Centric Question Answeringcited by 25
- 2021Disfl-QA: A Benchmark Dataset for Understanding Disfluencies in Question Answeringcited by 20
- 2025WavRAG: Audio-Integrated Retrieval Augmented Generation for Spoken Dialogue Modelscited by 18
38 more
- 2023HeySQuAD: A Spoken Question Answering Datasetcited by 16
- 2021Revisiting the Boundary between ASR and NLU in the Age of Conversational Dialog Systemscited by 15
- 2022DUAL: Discrete Spoken Unit Adaptive Learning for Textless Spoken Question Answeringcited by 15
- 2024Speech Retrieval-Augmented Generation without Automatic Speech Recognitioncited by 14
- 2023LibriSQA: A Novel Dataset and Framework for Spoken Question Answering with Large Language Modelscited by 13
- 2024SpeechDPR: End-to-End Spoken Passage Retrieval for Open-Domain Spoken Question Answeringcited by 12
- 2024Enhancing Low-Resource Language and Instruction Following Capabilities of Audio Language Modelscited by 10
- 2024On the Evaluation of Speech Foundation Models for Spoken Language Understandingcited by 9
- 2023GSQA: An End-to-End Model for Generative Spoken Question Answeringcited by 8
- 2024Towards ASR Robust Spoken Language Understanding Through In-Context Learning With Word Confusion Networkscited by 8
- 2024Contrastive Learning for Task-Independent SpeechLLM-Pretrainingcited by 8
- 2025SpokenNativQA: Multilingual Everyday Spoken Queries for LLMscited by 8
- 2024MERaLiON-AudioLLM: Bridging Audio and Language with Large Language Modelscited by 7
- 2025WearVox: An Egocentric Multichannel Voice Assistant Benchmark for Wearablescited by 7
- 2026Uncertainty Quantification for Multimodal Large Language Models with Incoherence-adjusted Semantic Volumecited by 7
- 2022On the Impact of Speech Recognition Errors in Passage Retrieval for Spoken Question Answeringcited by 6
- 2025The Role of Prosody in Spoken Question Answeringcited by 6
- 2025Measuring the Effect of Transcription Noise on Downstream Language Understanding Taskscited by 6
- 2022Video-Guided Curriculum Learning for Spoken Video Groundingcited by 5
- 2025NAVER LABS Europe Submission to the Instruction-following Trackcited by 5
- 2023MeeQA: Natural Questions in Meeting Transcriptscited by 4
- 2023The Interpreter Understands Your Meaning: End-to-end Spoken Language Understanding Aided by Speech Translationcited by 4
- 2024Multi-Modal Retrieval For Large Language Model Based Speech Recognitioncited by 4
- 2025KIT's Offline Speech Translation and Instruction Following Submission for IWSLT 2025cited by 4
- 2025IFEval-Audio: Benchmarking Instruction-Following Capability in Audio-based Large Language Modelscited by 4
- 2024Surprisingly Fragile: Assessing and Addressing Prompt Instability in Multimodal Foundation Modelscited by 3
- 2024Data-Centric Improvements for Enhancing Multi-Modal Understanding in Spoken Conversation Modelingcited by 3
- 2025NUTSHELL: A Dataset for Abstract Generation from Scientific Talkscited by 3
- 2026SpeechMapper: Speech-to-text Embedding Projector for LLMscited by 3
- 2022EVI: Multilingual Spoken Dialogue Tasks and Dataset for Knowledge-Based Enrolment, Verification, and Identificationcited by 2
- 2022Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Taskcited by 2
- 2024Zero-Shot End-To-End Spoken Question Answering In Medical Domaincited by 2
- 2024MoWE-Audio: Multitask AudioLLMs with Mixture of Weak Encoderscited by 2
- 2024A Multimodal Dense Retrieval Approach for Speech-Based Open-Domain Question Answeringcited by 2
- 2025Standard-to-Dialect Transfer Trends Differ across Text and Speech: A Case Study on Intent and Topic Classification in German Dialectscited by 2
- 2025End-to-end Contrastive Language-Speech Pretraining Model For Long-form Spoken Question Answeringcited by 2
- 2025Roles of MLLMs in Visually Rich Document Retrieval for RAG: A Surveycited by 2
- 2021An Initial Investigation of Non-Native Spoken Question-Answeringcited by 1
50 most cited of 74 All 74 (JSON)
- 2026DuplexSpeechBench-Document Grounding: Benchmarking Document Grounding and Hallucinations in Voice Agents29 Sep 2026
- 2026AS2D: Accelerating On-Demand Audio Understanding on Mobile Devices29 Sep 2026
- 2026CoSE-E: A Benchmark for Code-switched Speech Evaluation in Enterprise Settings28 Sep 2026
- 2026V=akQA: A Benchmark and Evaluation Study for Telugu Spoken Factoid Question Answering17 Sep 2026
- 2026VoiceTrace: A Benchmark and Retrieval Framework for Who-Said-What Speech Retrieval16 Sep 2026
- 2026The Bairong System for MLC-SLM 2026: Dynamic Question-Aware Evidence Routing for Multilingual Conversational Speech Understanding2 Sep 2026
- 2026ImageEval 2026: Culturally Grounded Arabic Multimodal Evaluation31 Aug 2026
- 2026Don't Just Listen, Try Planning: Graph-based Retrieval-Generation Agent for Long-form Audio Meeting Understanding25 Aug 2026
- 2026A Reranker for Orchestrating Heterogeneous Speech and Text Retrievers24 Aug 2026
- 2026Better Retrieval, Worse Robustness: How Multi-hop RAG Amplifies Upstream ASR Errors24 Aug 2026
- 2026INSPIRE: A Benchmark for Instruction-Aware Speech Retrieval17 Aug 2026
- 2026Should We Type or Talk to LLM Agents? A Comprehensive Study of Voice and Keyboard Input Perturbations4 Aug 2026
38 more
- 2026Hear, Invoke, and Understand: A Skill-Calling Multimodal Agent for Large Audio Language Models3 Aug 2026
- 2026At the Edge of Understanding: Sparse Autoencoders Trace The Limits of Transformer Generalization24 Jun 2026
- 2026PolySpeech-100: A Large-Scale Benchmark for Speech Understanding Across 100+ Languages and Dialects31 May 2026
- 2026ProactiveLLM: Learning Active Interaction for Streaming Large Language Models30 May 2026
- 2026KVoiceBench, KOpenAudioBench, and KMMAU: Agent-Driven Korean Speech Benchmarks for Evaluating SpeechLMs27 May 2026
- 2026ReasonAudio: A Benchmark for Evaluating Reasoning Beyond Matching in Text-Audio Retrieval5 May 2026
- 2026Attention-guided Evidence Grounding for Spoken Question Answering17 Mar 2026
- 2026Uncertainty Quantification for Multimodal Large Language Models with Incoherence-adjusted Semantic Volume27 Feb 2026
- 2026SpeechMapper: Speech-to-text Embedding Projector for LLMs28 Jan 2026
- 2025WearVox: An Egocentric Multichannel Voice Assistant Benchmark for Wearables25 Dec 2025
- 2025Roles of MLLMs in Visually Rich Document Retrieval for RAG: A Survey16 Dec 2025
- 2025End-to-end Contrastive Language-Speech Pretraining Model For Long-form Spoken Question Answering12 Nov 2025
- 2025Multimedia-Aware Question Answering: A Review of Retrieval and Cross-Modal Reasoning Architectures23 Oct 2025
- 2025Standard-to-Dialect Transfer Trends Differ across Text and Speech: A Case Study on Intent and Topic Classification in German Dialects9 Oct 2025
- 2025Instituto de Telecomunicacc~oes at IWSLT 2025: Aligning Small-Scale Speech and Language Models for Speech-to-Text Learning20 Jun 2025
- 2025NAVER LABS Europe Submission to the Instruction-following Track2 Jun 2025
- 2025Spoken question answering for visual queries29 May 2025
- 2025SpokenNativQA: Multilingual Everyday Spoken Queries for LLMs25 May 2025
- 2025IFEval-Audio: Benchmarking Instruction-Following Capability in Audio-based Large Language Models22 May 2025
- 2025KIT's Offline Speech Translation and Instruction Following Submission for IWSLT 202519 May 2025
- 2025NUTSHELL: A Dataset for Abstract Generation from Scientific Talks24 Feb 2025
- 2025WavRAG: Audio-Integrated Retrieval Augmented Generation for Spoken Dialogue Models20 Feb 2025
- 2025Measuring the Effect of Transcription Noise on Downstream Language Understanding Tasks19 Feb 2025
- 2025Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation12 Feb 2025
- 2025The Role of Prosody in Spoken Question Answering8 Feb 2025
- 2024Speech Retrieval-Augmented Generation without Automatic Speech Recognition21 Dec 2024
- 2024Data-Centric Improvements for Enhancing Multi-Modal Understanding in Spoken Conversation Modeling20 Dec 2024
- 2024Contrastive Learning for Task-Independent SpeechLLM-Pretraining20 Dec 2024
- 2024MERaLiON-AudioLLM: Bridging Audio and Language with Large Language Models13 Dec 2024
- 2024A Multimodal Dense Retrieval Approach for Speech-Based Open-Domain Question Answering20 Sep 2024
- 2024Enhancing Low-Resource Language and Instruction Following Capabilities of Audio Language Models17 Sep 2024
- 2024MoWE-Audio: Multitask AudioLLMs with Mixture of Weak Encoders10 Sep 2024
- 2024Surprisingly Fragile: Assessing and Addressing Prompt Instability in Multimodal Foundation Models26 Aug 2024
- 2024SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words19 Jun 2024
- 2024On the Evaluation of Speech Foundation Models for Spoken Language Understanding14 Jun 2024
- 2024Multi-Modal Retrieval For Large Language Model Based Speech Recognition13 Jun 2024
- 2024Zero-Shot End-To-End Spoken Question Answering In Medical Domain9 Jun 2024
- 2024VlogQA: Task, Dataset, and Baseline Models for Vietnamese Spoken-Based Machine Reading Comprehension5 Feb 2024
50 newest of 74 All 74 (JSON)
- 2025Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generationcited by 42
- 2025WavRAG: Audio-Integrated Retrieval Augmented Generation for Spoken Dialogue Modelscited by 18
- 2024Speech Retrieval-Augmented Generation without Automatic Speech Recognitioncited by 14
- 2025Roles of MLLMs in Visually Rich Document Retrieval for RAG: A Surveycited by 2
- 2025Multimedia-Aware Question Answering: A Review of Retrieval and Cross-Modal Reasoning Architecturescited by 0
- 2026A Reranker for Orchestrating Heterogeneous Speech and Text Retrieverscited by 0
- 2026A Reranker for Orchestrating Heterogeneous Speech and Text Retrievers24 Aug 2026
- 2025Roles of MLLMs in Visually Rich Document Retrieval for RAG: A Survey16 Dec 2025
- 2025Multimedia-Aware Question Answering: A Review of Retrieval and Cross-Modal Reasoning Architectures23 Oct 2025
- 2025WavRAG: Audio-Integrated Retrieval Augmented Generation for Spoken Dialogue Models20 Feb 2025
- 2025Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation12 Feb 2025
- 2024Speech Retrieval-Augmented Generation without Automatic Speech Recognition21 Dec 2024
- 2026KVoiceBench, KOpenAudioBench, and KMMAU: Agent-Driven Korean Speech Benchmarks for Evaluating SpeechLMscited by 0
- 2026Should We Type or Talk to LLM Agents? A Comprehensive Study of Voice and Keyboard Input Perturbationscited by 0
- 2026Don't Just Listen, Try Planning: Graph-based Retrieval-Generation Agent for Long-form Audio Meeting Understandingcited by 0
- 2026DuplexSpeechBench-Document Grounding: Benchmarking Document Grounding and Hallucinations in Voice Agentscited by 0
- 2026DuplexSpeechBench-Document Grounding: Benchmarking Document Grounding and Hallucinations in Voice Agents29 Sep 2026
- 2026Don't Just Listen, Try Planning: Graph-based Retrieval-Generation Agent for Long-form Audio Meeting Understanding25 Aug 2026
- 2026Should We Type or Talk to LLM Agents? A Comprehensive Study of Voice and Keyboard Input Perturbations4 Aug 2026
- 2026KVoiceBench, KOpenAudioBench, and KMMAU: Agent-Driven Korean Speech Benchmarks for Evaluating SpeechLMs27 May 2026
- 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
- 2024SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Wordscited by 67
- 2021Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answeringcited by 39
- 2022End-to-end Spoken Conversational Question Answering: Task, Dataset and Modelcited by 37
- 2021SD-QA: Spoken Dialectal Question Answering for the Real Worldcited by 36
- 2022SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Taskscited by 34
- 2021Self-supervised Dialogue Learning for Spoken Conversational Question Answeringcited by 27
- 2021M5Product: Self-harmonized Contrastive Learning for E-commercial Multi-modal Pretrainingcited by 26
- 2021NoiseQA: Challenge Set Evaluation for User-Centric Question Answeringcited by 25
- 2021Disfl-QA: A Benchmark Dataset for Understanding Disfluencies in Question Answeringcited by 20
- 2023HeySQuAD: A Spoken Question Answering Datasetcited by 16
- 2021Revisiting the Boundary between ASR and NLU in the Age of Conversational Dialog Systemscited by 15
12 most cited of 59 All 59 (JSON)
- 2026CoSE-E: A Benchmark for Code-switched Speech Evaluation in Enterprise Settings28 Sep 2026
- 2026V=akQA: A Benchmark and Evaluation Study for Telugu Spoken Factoid Question Answering17 Sep 2026
- 2026VoiceTrace: A Benchmark and Retrieval Framework for Who-Said-What Speech Retrieval16 Sep 2026
- 2026The Bairong System for MLC-SLM 2026: Dynamic Question-Aware Evidence Routing for Multilingual Conversational Speech Understanding2 Sep 2026
- 2026ImageEval 2026: Culturally Grounded Arabic Multimodal Evaluation31 Aug 2026
- 2026Better Retrieval, Worse Robustness: How Multi-hop RAG Amplifies Upstream ASR Errors24 Aug 2026
- 2026INSPIRE: A Benchmark for Instruction-Aware Speech Retrieval17 Aug 2026
- 2026Hear, Invoke, and Understand: A Skill-Calling Multimodal Agent for Large Audio Language Models3 Aug 2026
- 2026At the Edge of Understanding: Sparse Autoencoders Trace The Limits of Transformer Generalization24 Jun 2026
- 2026PolySpeech-100: A Large-Scale Benchmark for Speech Understanding Across 100+ Languages and Dialects31 May 2026
- 2026ProactiveLLM: Learning Active Interaction for Streaming Large Language Models30 May 2026
- 2026Attention-guided Evidence Grounding for Spoken Question Answering17 Mar 2026
12 newest of 59 All 59 (JSON)