# Annotation Tool and Dataset for Fact-Checking Podcasts

arXiv: https://arxiv.org/abs/2502.01402 · HTML: https://mlsanity.com/paper/2502.01402/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 19 references that are in the map.
- First posted on arXiv 3 Feb 2025.

## Builds on (most cited first, 10 of 10)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Web](https://mlsanity.com/paper/2305.13117/) (2023)
- [Powerset multi-class cross entropy loss for neural speaker diarization](https://mlsanity.com/paper/2310.13025/) (2023)
- [F-coref: Fast, Accurate and Easy to Use Coreference Resolution](https://mlsanity.com/paper/2209.04280/) (2022)
- [Surprising Efficacy of Fine-Tuned Transformers for Fact-Checking over Larger Language Models](https://mlsanity.com/paper/2402.12147/) (2024)
- [FactCheck Editor: Multilingual Text Editor with End-to-End fact-checking](https://mlsanity.com/paper/2404.19482/) (2024)
- [Detecting Check-Worthy Claims in Political Debates, Speeches, and Interviews Using Audio Data](https://mlsanity.com/paper/2306.05535/) (2023)
- [LiveFC: A System for Live Fact-Checking of Audio Streams](https://mlsanity.com/paper/2408.07448/) (2024)
- [Podcast Summary Assessment: A Resource for Evaluating Summary Assessment Methods](https://mlsanity.com/paper/2208.13265/) (2022)

## Built on by (most cited first, 2 of 2)

- [When Misinformation Speaks and Converses: Rethinking Fact-Checking in Audio Platforms](https://mlsanity.com/paper/2604.16767/) (2026)
- [TRILOGUE: A Trilingual Spoken Dialogue Fact-Checking Benchmark with Evidence and Paired Audio](https://mlsanity.com/paper/2609.04452/) (2026)

## Abstract (arXiv)

Podcasts are a popular medium on the web, featuring diverse and multilingual content that often includes unverified claims. Fact-checking podcasts is a challenging task, requiring transcription, annotation, and claim verification, all while preserving the contextual details of spoken content. Our tool offers a novel approach to tackle these challenges by enabling real-time annotation of podcasts during playback. This unique capability allows users to listen to the podcast and annotate key elements, such as check-worthy claims, claim spans, and contextual errors, simultaneously. By integrating advanced transcription models like OpenAI's Whisper and leveraging crowdsourced annotations, we create high-quality datasets to fine-tune multilingual transformer models such as XLM-RoBERTa for tasks like claim detection and stance classification. Furthermore, we release the annotated podcast transcripts and sample annotations with preliminary experiments.