# What Counts as Real? Speech Restoration and Voice Quality Conversion Pose New Challenges to Deepfake Detection

arXiv: https://arxiv.org/abs/2603.14033 · HTML: https://mlsanity.com/paper/2603.14033/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 14 Mar 2026.

## Builds on (most cited first, 26 of 26)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild](https://mlsanity.com/paper/2210.02437/) (2022)
- [ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale](https://mlsanity.com/paper/2408.08739/) (2024)
- [IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speech](https://mlsanity.com/paper/2506.21619/) (2025)
- [FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversion](https://mlsanity.com/paper/2210.15418/) (2022)
- [MLAAD: The Multi-Language Audio Anti-Spoofing Dataset](https://mlsanity.com/paper/2401.09512/) (2024)
- [The PartialSpoof Database and Countermeasures for the Detection of Short Fake Speech Segments Embedded in an Utterance](https://mlsanity.com/paper/2204.05177/) (2022)
- [VoxCPM: Tokenizer-Free TTS for Context-Aware Speech Generation and True-to-Life Voice Cloning](https://mlsanity.com/paper/2509.24650/) (2025)
- [FireRedTTS-2: Towards Long Conversational Speech Generation for Podcast and Chatbot](https://mlsanity.com/paper/2509.02020/) (2025)
- [ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matching](https://mlsanity.com/paper/2506.13053/) (2025)
- [An Initial Investigation for Detecting Partially Spoofed Audio](https://mlsanity.com/paper/2104.02518/) (2021)
- [Speech DF Arena: A Leaderboard for Speech DeepFake Detection Models](https://mlsanity.com/paper/2509.02859/) (2025)
- [CodecFake: Enhancing Anti-Spoofing Models Against Deepfake Audios from Codec-Based Speech Synthesis Systems](https://mlsanity.com/paper/2406.07237/) (2024)
- [PartialEdit: Identifying Partial Deepfakes in the Era of Neural Speech Editing](https://mlsanity.com/paper/2506.02958/) (2025)
- [Sidon: Fast and Robust Open-Source Multilingual Speech Restoration for Large-scale Dataset Cleansing](https://mlsanity.com/paper/2509.17052/) (2025)
- [Replay Attacks Against Audio Deepfake Detection](https://mlsanity.com/paper/2505.14862/) (2025)
- [TADA: Training-free Attribution and Out-of-Domain Detection of Audio Deepfakes](https://mlsanity.com/paper/2506.05802/) (2025)
- [VoXtream: Full-Stream Text-to-Speech with Extremely Low Latency](https://mlsanity.com/paper/2509.15969/) (2025)
- [Speech Synthesis along Perceptual Voice Quality Dimensions](https://mlsanity.com/paper/2501.08791/) (2025)
- [Towards Explainable Spoofed Speech Attribution and Detection:a Probabilistic Approach for Characterizing Speech Synthesizer Components](https://mlsanity.com/paper/2502.04049/) (2025)
- [Self Voice Conversion as an Attack against Neural Audio Watermarking](https://mlsanity.com/paper/2601.20432/) (2026)
- [Spoof Diarization: "What Spoofed When" in Partially Spoofed Audio](https://mlsanity.com/paper/2406.07816/) (2024)
- [A Survey of Threats Against Voice Authentication and Anti-Spoofing Systems](https://mlsanity.com/paper/2508.16843/) (2025)

## Abstract (arXiv)

Audio anti-spoofing systems are typically trained to assign one authenticity label to an entire speech utterance. This formulation becomes under-specified for transformations where the underlying speaker identity and linguistic content remain unchanged. We study this problem using benign, authenticity-preserving speech transformations, including voice quality conversion and speech restoration, applied to both bona fide and spoofed speech. Instead of treating all processed audio as spoofed, we factorise labels into source authenticity and processed status. Across SSL representations and DF-Arena fine-tuning experiments, we find that utterance processing status can transfer more reliably than source attribution: detectors can often identify that speech has been processed, while still confusing processed bona fide and processed spoofed speech. These results suggest that audio deepfake defences must move beyond the binary spoofed/authentic paradigm. Robust detection requires granular reporting on source authenticity, processing status, and precise processing localisation.