# Contextualized Attention-based Knowledge Transfer for Spoken Conversational Question Answering

arXiv: https://arxiv.org/abs/2010.11066 · HTML: https://mlsanity.com/paper/2010.11066/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2020.

## Built on by (most cited first, 27 of 27)

- [SimCVD: Simple Contrastive Voxel-Wise Representation Distillation for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2108.06227/) (2021)
- [Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation](https://mlsanity.com/paper/2401.01044/) (2024)
- [Momentum Contrastive Voxel-wise Representation Learning for Semi-supervised Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2105.07059/) (2021)
- [Auto-Encoding Knowledge Graph for Unsupervised Medical Report Generation](https://mlsanity.com/paper/2111.04318/) (2021)
- [Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answering](https://mlsanity.com/paper/2109.03381/) (2021)
- [End-to-end Spoken Conversational Question Answering: Task, Dataset and Model](https://mlsanity.com/paper/2204.14272/) (2022)
- [Why Prompt Design Matters and Works: A Complexity Analysis of Prompt Search Space in LLMs](https://mlsanity.com/paper/2503.10084/) (2025)
- [Self-supervised Dialogue Learning for Spoken Conversational Question Answering](https://mlsanity.com/paper/2106.02182/) (2021)
- [HeySQuAD: A Spoken Question Answering Dataset](https://mlsanity.com/paper/2304.13689/) (2023)
- [Aligning Source Visual and Target Language Domains for Unpaired Video Captioning](https://mlsanity.com/paper/2211.12148/) (2022)
- [From Good to Best: Two-Stage Training for Cross-lingual Machine Reading Comprehension](https://mlsanity.com/paper/2112.04735/) (2021)
- [A Decentralized Reinforcement Learning Framework for Efficient Passage of Emergency Vehicles](https://mlsanity.com/paper/2111.00278/) (2021)
- [EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control System](https://mlsanity.com/paper/2206.13441/) (2022)
- [MEGAN: Memory Enhanced Graph Attention Network for Space-Time Video Super-Resolution](https://mlsanity.com/paper/2110.15327/) (2021)
- [Bridging the Gap between Language Models and Cross-Lingual Sequence Labeling](https://mlsanity.com/paper/2204.05210/) (2022)
- [VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning](https://mlsanity.com/paper/2511.17731/) (2025)
- [Video-Guided Curriculum Learning for Spoken Video Grounding](https://mlsanity.com/paper/2209.00277/) (2022)
- [UniHGKR: Unified Instruction-aware Heterogeneous Knowledge Retrievers](https://mlsanity.com/paper/2410.20163/) (2024)
- [Enabling All In-Edge Deep Learning: A Literature Review](https://mlsanity.com/paper/2204.03326/) (2022)
- [Computation and Parameter Efficient Multi-Modal Fusion Transformer for Cued Speech Recognition](https://mlsanity.com/paper/2401.17604/) (2024)
- [ELP-Adapters: Parameter Efficient Adapter Tuning for Various Speech Processing Tasks](https://mlsanity.com/paper/2407.21066/) (2024)
- [The End of Reward Engineering: How LLMs Are Redefining Multi-Agent Coordination](https://mlsanity.com/paper/2601.08237/) (2026)
- [Calibrate and Refine! A Novel and Agile Framework for ASR-error Robust Intent Detection](https://mlsanity.com/paper/2205.11008/) (2022)
- [Low-resource Accent Classification in Geographically-proximate Settings: A Forensic and Sociophonetics Perspective](https://mlsanity.com/paper/2206.12759/) (2022)
- [EMVLight: A Decentralized Reinforcement Learning Framework for Efficient Passage of Emergency Vehicles](https://mlsanity.com/paper/2109.05429/) (2021)
- [Exploring and Exploiting Multi-Granularity Representations for Machine Reading Comprehension](https://mlsanity.com/paper/2208.08750/) (2022)
- [When LLM Meets Tree Search: A Systematic View of Inference as Search in Large Language Models](https://mlsanity.com/paper/2608.30395/) (2026)

## Abstract (arXiv)

Spoken conversational question answering (SCQA) requires machines to model complex dialogue flow given the speech utterances and text corpora. Different from traditional text question answering (QA) tasks, SCQA involves audio signal processing, passage comprehension, and contextual understanding. However, ASR systems introduce unexpected noisy signals to the transcriptions, which result in performance degradation on SCQA. To overcome the problem, we propose CADNet, a novel contextualized attention-based distillation approach, which applies both cross-attention and self-attention to obtain ASR-robust contextualized embedding representations of the passage and dialogue history for performance improvements. We also introduce the spoken conventional knowledge distillation framework to distill the ASR-robust knowledge from the estimated probabilities of the teacher model to the student. We conduct extensive experiments on the Spoken-CoQA dataset and demonstrate that our approach achieves remarkable performance in this task.