# On the Impact of Speech Recognition Errors in Passage Retrieval for Spoken Question Answering

arXiv: https://arxiv.org/abs/2209.12944 · HTML: https://mlsanity.com/paper/2209.12944/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 17 references that are in the map.
- First posted on arXiv 26 Sep 2022.

## Builds on (most cited first, 13 of 13)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [SLURP: A Spoken Language Understanding Resource Package](https://mlsanity.com/paper/2011.13205/) (2020)
- [MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages](https://mlsanity.com/paper/2204.08582/) (2022)
- [Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehension](https://mlsanity.com/paper/1804.00320/) (2018)
- [SD-QA: Spoken Dialectal Question Answering for the Real World](https://mlsanity.com/paper/2109.12072/) (2021)
- [NoiseQA: Challenge Set Evaluation for User-Centric Question Answering](https://mlsanity.com/paper/2102.08345/) (2021)
- [Dealing with Typos for BERT-based Passage Retrieval and Ranking](https://mlsanity.com/paper/2108.12139/) (2021)
- [Analysing the Robustness of Dual Encoders for Dense Retrieval Against Misspellings](https://mlsanity.com/paper/2205.02303/) (2022)
- [CharacterBERT and Self-Teaching for Improving the Robustness of Dense Retrievers on Queries with Typos](https://mlsanity.com/paper/2204.00716/) (2022)
- [SANTLR: Speech Annotation Toolkit for Low Resource Languages](https://mlsanity.com/paper/1908.01067/) (2019)

## Built on by (most cited first, 6 of 6)

- [Noise-Robust Dense Retrieval via Contrastive Alignment Post Training](https://mlsanity.com/paper/2304.03401/) (2023)
- [A Multimodal Dense Retrieval Approach for Speech-Based Open-Domain Question Answering](https://mlsanity.com/paper/2409.13483/) (2024)
- [Towards Proactive Information Retrieval in Noisy Text with Wikipedia Concepts](https://mlsanity.com/paper/2210.09877/) (2022)
- [On Monotonic Aggregation for Open-domain QA](https://mlsanity.com/paper/2308.04176/) (2023)
- [SQuTR: A Robustness Benchmark for Spoken Query to Text Retrieval under Acoustic Noise](https://mlsanity.com/paper/2602.12783/) (2026)
- [LuxSQA: Ask Me in Luxembourgish with TTS-Augmented Spoken Question Answering](https://mlsanity.com/paper/2607.02763/) (2026)

## Abstract (arXiv)

Interacting with a speech interface to query a Question Answering (QA) system is becoming increasingly popular. Typically, QA systems rely on passage retrieval to select candidate contexts and reading comprehension to extract the final answer. While there has been some attention to improving the reading comprehension part of QA systems against errors that automatic speech recognition (ASR) models introduce, the passage retrieval part remains unexplored. However, such errors can affect the performance of passage retrieval, leading to inferior end-to-end performance. To address this gap, we augment two existing large-scale passage ranking and open domain QA datasets with synthetic ASR noise and study the robustness of lexical and dense retrievers against questions with ASR noise. Furthermore, we study the generalizability of data augmentation techniques across different domains; with each domain being a different language dialect or accent. Finally, we create a new dataset with questions voiced by human users and use their transcriptions to show that the retrieval performance can further degrade when dealing with natural ASR noise instead of synthetic ASR noise.