mlsanityData to 7 Oct 2026

Knowledge Distillation for Improved Accuracy in Spoken Question Answering

arXiv 2010.11067 ↗First posted 21 Oct 2020cs.CL cs.AI cs.IRcited by 31

Not on the map. Why?

Abstract (arXiv)

Spoken question answering (SQA) is a challenging task that requires the machine to fully understand the complex spoken documents. Automatic speech recognition (ASR) plays a significant role in the development of QA systems. However, the recent work shows that ASR systems generate highly noisy transcripts, which critically limit the capability of machine comprehension on the SQA task. To address the issue, we present a novel distillation framework. Specifically, we devise a training strategy to perform knowledge distillation (KD) from spoken documents and written counterparts. Our work makes a step towards distilling knowledge from the language model as a supervision signal to lead to better student accuracy by reducing the misalignment between automatic and manual transcriptions. Experiments demonstrate that our approach outperforms several state-of-the-art language models on the Spoken-SQuAD dataset.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 6; 2023: 5; 2024: 7; 2025: 3; 2026: 1.

31 papers.

Sort:

By area:

  1. 2021SimCVD: Simple Contrastive Voxel-Wise Representation Distillation for Semi-Supervised Medical Image Segmentationcited by 86
  2. 2024Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generationcited by 54
  3. 2021Momentum Contrastive Voxel-wise Representation Learning for Semi-supervised Volumetric Medical Image Segmentationcited by 49
  4. 2021Auto-Encoding Knowledge Graph for Unsupervised Medical Report Generationcited by 47
  5. 2021Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answeringcited by 39
  6. 2022End-to-end Spoken Conversational Question Answering: Task, Dataset and Modelcited by 37
  7. 2025Why Prompt Design Matters and Works: A Complexity Analysis of Prompt Search Space in LLMscited by 29
  8. 2021Self-supervised Dialogue Learning for Spoken Conversational Question Answeringcited by 27
  9. 2021Maximum Bayes Smatch Ensemble Distillation for AMR Parsingcited by 27
  10. 2023HeySQuAD: A Spoken Question Answering Datasetcited by 16
  11. 2022Aligning Source Visual and Target Language Domains for Unpaired Video Captioningcited by 14
  12. 2022EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control Systemcited by 9
19 more
  1. 2021MEGAN: Memory Enhanced Graph Attention Network for Space-Time Video Super-Resolutioncited by 8
  2. 2021A Decentralized Reinforcement Learning Framework for Efficient Passage of Emergency Vehiclescited by 8
  3. 2023GSQA: An End-to-End Model for Generative Spoken Question Answeringcited by 8
  4. 2022Bridging the Gap between Language Models and Cross-Lingual Sequence Labelingcited by 7
  5. 2025Measuring the Effect of Transcription Noise on Downstream Language Understanding Taskscited by 6
  6. 2022Graph-in-Graph Network for Automatic Gene Ontology Description Generationcited by 5
  7. 2024A Variance-Preserving Interpolation Approach for Diffusion Models with Applications to Single Channel Speech Enhancement and Recognitioncited by 5
  8. 2024UniHGKR: Unified Instruction-aware Heterogeneous Knowledge Retrieverscited by 5
  9. 2023Leveraging Pretrained ASR Encoders for Effective and Efficient End-to-End Speech Intent Classification and Slot Fillingcited by 3
  10. 2023Deep Active Audio Feature Learning in Resource-Constrained Environmentscited by 3
  11. 2024Computation and Parameter Efficient Multi-Modal Fusion Transformer for Cued Speech Recognitioncited by 3
  12. 2024ELP-Adapters: Parameter Efficient Adapter Tuning for Various Speech Processing Taskscited by 3
  13. 2026The End of Reward Engineering: How LLMs Are Redefining Multi-Agent Coordinationcited by 3
  14. 2024Zero-Shot End-To-End Spoken Question Answering In Medical Domaincited by 2
  15. 2024A Multimodal Dense Retrieval Approach for Speech-Based Open-Domain Question Answeringcited by 2
  16. 2022Calibrate and Refine! A Novel and Agile Framework for ASR-error Robust Intent Detectioncited by 1
  17. 2023Natural Response Generation for Chinese Reading Comprehensioncited by 1
  18. 2025Spoken question answering for visual queriescited by 1
  19. 2021EMVLight: A Decentralized Reinforcement Learning Framework for Efficient Passage of Emergency Vehiclescited by 0