mlsanityData to 7 Oct 2026

SpeechDPR: End-to-End Spoken Passage Retrieval for Open-Domain Spoken Question Answering

arXiv 2401.13463 ↗First posted 24 Jan 2024cs.CL cs.IR cs.SDcited by 12

Not on the map. Why?

Abstract (arXiv)

Spoken Question Answering (SQA) is essential for machines to reply to user's question by finding the answer span within a given spoken passage. SQA has been previously achieved without ASR to avoid recognition errors and Out-of-Vocabulary (OOV) problems. However, the real-world problem of Open-domain SQA (openSQA), in which the machine needs to first retrieve passages that possibly contain the answer from a spoken archive in addition, was never considered. This paper proposes the first known end-to-end framework, Speech Dense Passage Retriever (SpeechDPR), for the retrieval component of the openSQA problem. SpeechDPR learns a sentence-level semantic representation by distilling knowledge from the cascading model of unsupervised ASR (UASR) and text dense retriever (TDR). No manually transcribed speech data is needed. Initial experiments showed performance comparable to the cascading model of UASR and TDR, and significantly better when UASR was poor, verifying this approach is more robust to speech recognition errors.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 20 of its 36 references that are in the map. First posted on arXiv 24 Jan 2024.

Builds on

20 of 36 references.

References · 20

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  5. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  6. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  7. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  8. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  9. 2021Generative Spoken Language Modeling from Raw Audiocited by 336
  10. 2019End-to-End Open-Domain Question Answering with BERTserinicited by 152
  11. 2021Direct speech-to-speech translation with discrete unitscited by 148
  12. 2021Unsupervised Speech Recognitioncited by 147
8 more
  1. 2018Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehensioncited by 74
  2. 2022Towards End-to-end Unsupervised Speech Recognitioncited by 44
  3. 2022SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Taskscited by 34
  4. 2018Vision as an Interlingua: Learning Multilingual Semantic Embeddings of Untranscribed Speechcited by 26
  5. 2018ODSQA: Open-domain Spoken Question Answering Datasetcited by 21
  6. 2017Semantic speech retrieval with a visually grounded model of untranscribed speechcited by 20
  7. 2019Neural Network based End-to-End Query by Example Spoken Term Detectioncited by 8
  8. 2017Query-by-example Spoken Term Detection using Attention-based Multi-hop Networkscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 6.

12 papers.

  1. 2024Speech Retrieval-Augmented Generation without Automatic Speech Recognitioncited by 14
  2. 2024Retrieval-Enhanced Machine Learning: Synthesis and Opportunitiescited by 11
  3. 2024Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration Approachcited by 3
  4. 2025VoxRAG: A Step Toward Transcription-Free RAG Systems in Spoken Question Answeringcited by 3
  5. 2026SQuTR: A Robustness Benchmark for Spoken Query to Text Retrieval under Acoustic Noisecited by 1
  6. 2026From Awareness to Adherence: Bridging the Context Gap in Spoken Dialogue Systems via Context-Aware Decodingcited by 1
  7. 2025Rhapsody: A Dataset for Highlight Detection in Podcastscited by 0
  8. 2025Expressive Speech Retrieval using Natural Language Descriptions of Speaking Stylecited by 0
  9. 2026LuxSQA: Ask Me in Luxembourgish with TTS-Augmented Spoken Question Answeringcited by 0
  10. 2026INSPIRE: A Benchmark for Instruction-Aware Speech Retrievalcited by 0
  11. 2026Don't Just Listen, Try Planning: Graph-based Retrieval-Generation Agent for Long-form Audio Meeting Understandingcited by 0
  12. 2026VoiceTrace: A Benchmark and Retrieval Framework for Who-Said-What Speech Retrievalcited by 0