# Self-supervised Dialogue Learning for Spoken Conversational Question Answering

arXiv: https://arxiv.org/abs/2106.02182 · HTML: https://mlsanity.com/paper/2106.02182/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 27 references that are in the map.
- First posted on arXiv 4 Jun 2021.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Bidirectional Attention Flow for Machine Comprehension](https://mlsanity.com/paper/1611.01603/) (2016)
- [MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification](https://mlsanity.com/paper/2004.12239/) (2020)
- [Neural Approaches to Conversational AI](https://mlsanity.com/paper/1809.08267/) (2018)
- [Towards end-to-end spoken language understanding](https://mlsanity.com/paper/1802.08395/) (2018)
- [CT Super-resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble(GAN-CIRCLE)](https://mlsanity.com/paper/1808.04256/) (2018)
- [Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehension](https://mlsanity.com/paper/1804.00320/) (2018)
- [Momentum Contrastive Voxel-wise Representation Learning for Semi-supervised Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2105.07059/) (2021)
- [Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection](https://mlsanity.com/paper/2009.04703/) (2020)
- [Towards Data Distillation for End-to-end Spoken Conversational Question Answering](https://mlsanity.com/paper/2010.08923/) (2020)
- [Knowledge Distillation for Improved Accuracy in Spoken Question Answering](https://mlsanity.com/paper/2010.11067/) (2020)
- [SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering](https://mlsanity.com/paper/1812.03593/) (2018)
- [Unsupervised Wasserstein Distance Guided Domain Adaptation for 3D Multi-Domain Liver Segmentation](https://mlsanity.com/paper/2009.02831/) (2020)
- [Contextualized Attention-based Knowledge Transfer for Spoken Conversational Question Answering](https://mlsanity.com/paper/2010.11066/) (2020)
- [FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension](https://mlsanity.com/paper/1711.07341/) (2017)
- [Structure-sensitive Multi-scale Deep Neural Network for Low-Dose CT Denoising](https://mlsanity.com/paper/1805.00587/) (2018)
- [Adaptive Bi-directional Attention: Exploring Multi-Granularity Representations for Machine Reading Comprehension](https://mlsanity.com/paper/2012.10877/) (2020)
- [ODSQA: Open-domain Spoken Question Answering Dataset](https://mlsanity.com/paper/1808.02280/) (2018)
- [Self-Supervised Dialogue Learning](https://mlsanity.com/paper/1907.00448/) (2019)
- [Low-Dose CT via Deep CNN with Skip Connection and Network in Network](https://mlsanity.com/paper/1811.10564/) (2018)
- [An Audio-enriched BERT-based Framework for Spoken Multiple-choice Question Answering](https://mlsanity.com/paper/2005.12142/) (2020)

## Built on by (most cited first, 28 of 28)

- [SimCVD: Simple Contrastive Voxel-Wise Representation Distillation for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2108.06227/) (2021)
- [Auto-Encoding Knowledge Graph for Unsupervised Medical Report Generation](https://mlsanity.com/paper/2111.04318/) (2021)
- [Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answering](https://mlsanity.com/paper/2109.03381/) (2021)
- [End-to-end Spoken Conversational Question Answering: Task, Dataset and Model](https://mlsanity.com/paper/2204.14272/) (2022)
- [Physics-Driven Diffusion Models for Impact Sound Synthesis from Videos](https://mlsanity.com/paper/2303.16897/) (2023)
- [Large Language Models Meet Harry Potter: A Bilingual Dataset for Aligning Dialogue Agents with Characters](https://mlsanity.com/paper/2211.06869/) (2022)
- [HeySQuAD: A Spoken Question Answering Dataset](https://mlsanity.com/paper/2304.13689/) (2023)
- [Alleviating Over-smoothing for Unsupervised Sentence Representation](https://mlsanity.com/paper/2305.06154/) (2023)
- [A Decentralized Reinforcement Learning Framework for Efficient Passage of Emergency Vehicles](https://mlsanity.com/paper/2111.00278/) (2021)
- [EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control System](https://mlsanity.com/paper/2206.13441/) (2022)
- [MEGAN: Memory Enhanced Graph Attention Network for Space-Time Video Super-Resolution](https://mlsanity.com/paper/2110.15327/) (2021)
- [Bridging the Gap between Language Models and Cross-Lingual Sequence Labeling](https://mlsanity.com/paper/2204.05210/) (2022)
- [VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning](https://mlsanity.com/paper/2511.17731/) (2025)
- [Video-Guided Curriculum Learning for Spoken Video Grounding](https://mlsanity.com/paper/2209.00277/) (2022)
- [A Variance-Preserving Interpolation Approach for Diffusion Models with Applications to Single Channel Speech Enhancement and Recognition](https://mlsanity.com/paper/2405.16952/) (2024)
- [UniHGKR: Unified Instruction-aware Heterogeneous Knowledge Retrievers](https://mlsanity.com/paper/2410.20163/) (2024)
- [Computation and Parameter Efficient Multi-Modal Fusion Transformer for Cued Speech Recognition](https://mlsanity.com/paper/2401.17604/) (2024)
- [ELP-Adapters: Parameter Efficient Adapter Tuning for Various Speech Processing Tasks](https://mlsanity.com/paper/2407.21066/) (2024)
- [The End of Reward Engineering: How LLMs Are Redefining Multi-Agent Coordination](https://mlsanity.com/paper/2601.08237/) (2026)
- [Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Task](https://mlsanity.com/paper/2212.03866/) (2022)
- [NeuroSonic: Conditional Flow Matching for EEG-to-Speech Reconstruction](https://mlsanity.com/paper/2606.24087/) (2026)
- [Calibrate and Refine! A Novel and Agile Framework for ASR-error Robust Intent Detection](https://mlsanity.com/paper/2205.11008/) (2022)
- [Low-resource Accent Classification in Geographically-proximate Settings: A Forensic and Sociophonetics Perspective](https://mlsanity.com/paper/2206.12759/) (2022)
- [Orca: A Few-shot Benchmark for Chinese Conversational Machine Reading Comprehension](https://mlsanity.com/paper/2302.13619/) (2023)
- [EMVLight: A Decentralized Reinforcement Learning Framework for Efficient Passage of Emergency Vehicles](https://mlsanity.com/paper/2109.05429/) (2021)
- [Exploring and Exploiting Multi-Granularity Representations for Machine Reading Comprehension](https://mlsanity.com/paper/2208.08750/) (2022)
- [LuxSQA: Ask Me in Luxembourgish with TTS-Augmented Spoken Question Answering](https://mlsanity.com/paper/2607.02763/) (2026)
- [When LLM Meets Tree Search: A Systematic View of Inference as Search in Large Language Models](https://mlsanity.com/paper/2608.30395/) (2026)

## Abstract (arXiv)

In spoken conversational question answering (SCQA), the answer to the corresponding question is generated by retrieving and then analyzing a fixed spoken document, including multi-part conversations. Most SCQA systems have considered only retrieving information from ordered utterances. However, the sequential order of dialogue is important to build a robust spoken conversational question answering system, and the changes of utterances order may severely result in low-quality and incoherent corpora. To this end, we introduce a self-supervised learning approach, including incoherence discrimination, insertion detection, and question prediction, to explicitly capture the coreference resolution and dialogue coherence among spoken documents. Specifically, we design a joint learning framework where the auxiliary self-supervised tasks can enable the pre-trained SCQA systems towards more coherent and meaningful spoken dialogue learning. We also utilize the proposed self-supervised learning tasks to capture intra-sentence coherence. Experimental results demonstrate that our proposed method provides more coherent, meaningful, and appropriate responses, yielding superior performance gains compared to the original pre-trained language models. Our method achieves state-of-the-art results on the Spoken-CoQA dataset.