# Context-Aware Multimodal Claim Verification in Spoken Dialogues

arXiv: https://arxiv.org/abs/2606.11420 · HTML: https://mlsanity.com/paper/2606.11420/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 34 references that are in the map.
- First posted on arXiv 9 Jun 2026.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [WhisperX: Time-Accurate Speech Transcription of Long-Form Audio](https://mlsanity.com/paper/2303.00747/) (2023)
- [End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Models](https://mlsanity.com/paper/2205.12487/) (2022)
- [Fine-grained Fact Verification with Kernel Graph Attention Network](https://mlsanity.com/paper/1910.09796/) (2019)
- [GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification](https://mlsanity.com/paper/1908.01843/) (2019)
- [DialFact: A Benchmark for Fact-Checking in Dialogue](https://mlsanity.com/paper/2110.08222/) (2021)
- [A Large-Scale Evaluation of Speech Foundation Models](https://mlsanity.com/paper/2404.09385/) (2024)
- [MoonCast: High-Quality Zero-Shot Podcast Generation](https://mlsanity.com/paper/2503.14345/) (2025)
- [Detecting Deception in Political Debates Using Acoustic and Textual Features](https://mlsanity.com/paper/1910.01990/) (2019)
- [Detecting Check-Worthy Claims in Political Debates, Speeches, and Interviews Using Audio Data](https://mlsanity.com/paper/2306.05535/) (2023)
- [Mapping the Podcast Ecosystem with the Structured Podcast Research Corpus](https://mlsanity.com/paper/2411.07892/) (2024)
- [Automated Fact-Checking in Dialogue: Are Specialized Models Needed?](https://mlsanity.com/paper/2311.08195/) (2023)
- [LiveFC: A System for Live Fact-Checking of Audio Streams](https://mlsanity.com/paper/2408.07448/) (2024)
- [TS-SUPERB: A Target Speech Processing Benchmark for Speech Self-Supervised Learning Models](https://mlsanity.com/paper/2505.06660/) (2025)
- [MAD: A Benchmark for Multi-Turn Audio Dialogue Fact-Checking](https://mlsanity.com/paper/2508.12186/) (2025)

## Built on by (most cited first, 1 of 1)

- [TRILOGUE: A Trilingual Spoken Dialogue Fact-Checking Benchmark with Evidence and Paired Audio](https://mlsanity.com/paper/2609.04452/) (2026)

## Abstract (arXiv)

Spoken factual claims often occur within multi-turn conversations, where surrounding dialogue can provide context unavailable from the claim alone. Yet most fact-checking research evaluates isolated text, leaving conversational audio under-studied. We introduce MAD2, a synthetic Multi-turn Audio Dialogues benchmark for spoken claim verification with 1,000 two-speaker dialogues, 1,230 sentence-level check-worthy candidate annotations, and approximately 10 hours of audio. We also propose calibrated multimodal fusion of a context-aware audio encoder and a dialogue-aware text model. Adding dialogue context improves verification across settings, although the gains vary by scenario. Past-only context often approaches local offline performance, suggesting its usefulness when future context is unavailable. Fusion achieves its highest mean advantage over text with full-dialogue context, but does not consistently or significantly outperform text across settings. Under full-dialogue context, exploratory subgroup results show greater performance variation across dialogue scenarios for text and fusion, but across spread styles for audio.