# Spoof Diarization: "What Spoofed When" in Partially Spoofed Audio

arXiv: https://arxiv.org/abs/2406.07816 · HTML: https://mlsanity.com/paper/2406.07816/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 12 Jun 2024.

## Builds on (most cited first, 17 of 17)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Pay Attention to MLPs](https://mlsanity.com/paper/2105.08050/) (2021)
- [ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection](https://mlsanity.com/paper/2109.00537/) (2021)
- [A Review of Speaker Diarization: Recent Advances with Deep Learning](https://mlsanity.com/paper/2101.09624/) (2021)
- [ADD 2022: the First Audio Deep Synthesis Detection Challenge](https://mlsanity.com/paper/2202.08433/) (2022)
- [AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset](https://mlsanity.com/paper/2311.15308/) (2023)
- [ASVspoof 2019: spoofing countermeasures for the detection of synthesized, converted and replayed speech](https://mlsanity.com/paper/2102.05889/) (2021)
- [The PartialSpoof Database and Countermeasures for the Detection of Short Fake Speech Segments Embedded in an Utterance](https://mlsanity.com/paper/2204.05177/) (2022)
- [Half-Truth: A Partially Fake Audio Detection Dataset](https://mlsanity.com/paper/2104.03617/) (2021)
- [Towards Open-Set Object Detection and Discovery](https://mlsanity.com/paper/2204.05604/) (2022)
- [The Second DIHARD Diarization Challenge: Dataset, task, and baselines](https://mlsanity.com/paper/1906.07839/) (2019)
- [The Vicomtech Audio Deepfake Detection System based on Wav2Vec2 for the 2022 ADD Challenge](https://mlsanity.com/paper/2203.01573/) (2022)
- [An Initial Investigation for Detecting Partially Spoofed Audio](https://mlsanity.com/paper/2104.02518/) (2021)
- [Multi-Task Learning in Utterance-Level and Segmental-Level Spoof Detection](https://mlsanity.com/paper/2107.14132/) (2021)
- [Waveform Boundary Detection for Partially Spoofed Audio](https://mlsanity.com/paper/2211.00226/) (2022)
- [Partially Fake Audio Detection by Self-attention-based Fake Span Discovery](https://mlsanity.com/paper/2202.06684/) (2022)
- [Range-Based Equal Error Rate for Spoof Localization](https://mlsanity.com/paper/2305.17739/) (2023)

## Built on by (most cited first, 3 of 3)

- [Multi-level SSL Feature Gating for Audio Deepfake Detection](https://mlsanity.com/paper/2509.03409/) (2025)
- [What Counts as Real? Speech Restoration and Voice Quality Conversion Pose New Challenges to Deepfake Detection](https://mlsanity.com/paper/2603.14033/) (2026)
- [From Scores to Evidence: Auditable Decisions Can Improve Speech Deepfake Detection](https://mlsanity.com/paper/2609.08899/) (2026)

## Abstract (arXiv)

This paper defines Spoof Diarization as a novel task in the Partial Spoof (PS) scenario. It aims to determine what spoofed when, which includes not only locating spoof regions but also clustering them according to different spoofing methods. As a pioneering study in spoof diarization, we focus on defining the task, establishing evaluation metrics, and proposing a benchmark model, namely the Countermeasure-Condition Clustering (3C) model. Utilizing this model, we first explore how to effectively train countermeasures to support spoof diarization using three labeling schemes. We then utilize spoof localization predictions to enhance the diarization performance. This first study reveals the high complexity of the task, even in restricted scenarios where only a single speaker per audio file and an oracle number of spoofing methods are considered. Our code is available at https://github.com/nii-yamagishilab/PartialSpoof.