mlsanityData to 14 Oct 2026

To Trust or Not to Trust: Retrieval-Augmented Fact Checking in Speech

arXiv 2609.30227 ↗First posted 24 Sep 2026cs.LG cs.AI cs.CLcited by 0

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval also fits: Conversational and speech RAG

Abstract (arXiv)

Online misinformation increasingly appears in spoken formats such as news clips, podcasts, interviews, political speeches, and social media videos, creating a need for fact-checking systems that can verify claims directly from speech. We introduce VeriSpeak, a probe benchmark for studying speech-based fact verification in Large Audio Language Models (LALMs). VeriSpeak contains 3,879 spoken claims spanning temporal, geographical, and relational facts, with balanced true and false labels. The benchmark is designed to examine whether factual verification ability transfers from text to speech, and whether retrieval-augmented LALMs can use textual evidence to correctly support or refute spoken claims. Our experiments reveal a consistent text-speech modality gap: LALMs that verify written claims reliably often fail on the same claims when spoken. Moreover, retrieval alone provides limited gains because models frequently conflate retrieved evidence with the spoken claim. In contrast, retrieval combined with explicit reasoning improves claim-evidence comparison, with a thinking-tuned LALM reaching 86.1% accuracy. VeriSpeak highlights that effective speech misinformation detection requires not only speech understanding, but also grounded reasoning over retrieved evidence. The dataset is publicly available via Hugging Face at https://huggingface.co/datasets/abhiram4572/VeriSpeak.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 27 of its 45 references that are in the map. First posted on arXiv 24 Sep 2026.

Builds on

27 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection17 Oct

Other references · 26

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  5. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  6. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  7. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  8. 2020Fact or Fiction: Verifying Scientific Claimscited by 554
  9. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
  10. 2023Pengi: An Audio Language Model for Audio Taskscited by 220
  11. 2020HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verificationcited by 201
  12. 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 186
14 more
  1. 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilitiescited by 156
  2. 2022End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Modelscited by 107
  3. 2025Words or Vision: Do Vision-Language Models Have Blind Faith in Text?cited by 89
  4. 2023Multimodal Automated Fact-Checking: A Surveycited by 53
  5. 2024RAGAR, Your Falsehood Radar: RAG-Augmented Reasoning for Political Fact-Checking using Multimodal Large Language Modelscited by 27
  6. 2024Bridge the Modality and Capability Gaps in Vision-Language Model Selectioncited by 20
  7. 2025Closing the Gap Between Text and Speech Understanding in LLMscited by 17
  8. 2023FACTIFY3M: A Benchmark for Multimodal Fact Verification with Explainability through 5W Question-Answeringcited by 16
  9. 2025Understanding the Modality Gap: An Empirical Study on the Speech-Text Alignment Mechanism of Large Speech Language Modelscited by 13
  10. 2025See-Saw Modality Balance: See Gradient, and Sew Impaired Vision-Language Balance to Mitigate Dominant Modality Biascited by 9
  11. 2025Modality-Balancing Preference Optimization of Large Multimodal Models by Adversarial Negative Miningcited by 8
  12. 2022COFAR: Commonsense and Factual Reasoning in Image Searchcited by 4
  13. 2024Visual Text Matters: Improving Text-KVQA with Visual Text Entity Knowledge-aware Large Multimodal Assistantcited by 3
  14. 2025DTW-Align: Bridging the Modality Gap in End-to-End Speech Translation with Dynamic Time Warping Alignmentcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Retrieval Augmented Fact Verification by Synthesizing Contrastive Arguments
  2. 2025Beyond Retrieval: Improving Evidence Quality for LLM-based Multimodal Fact-Checking
  3. 2025Resolving Conflicting Evidence in Automated Fact-Checking: A Study on Retrieval-Augmented LLMs
  4. 2024AFaCTA: Assisting the Annotation of Factual Claim Detection with Reliable LLM Annotators
  5. 2024Reinforcement Retrieval Leveraging Fine-grained Feedback for Fact Checking News Claims with Black-Box LLM
  6. 2026Context Shapes LLMs Retrieval-Augmented Fact-Checking Effectiveness
  7. 2025Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals
  8. 2024FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation