# Lightweight Model Attribution and Detection of Synthetic Speech via Audio Residual Fingerprints

arXiv: https://arxiv.org/abs/2411.14013 · HTML: https://mlsanity.com/paper/2411.14013/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 77 references that are in the map.
- First posted on arXiv 21 Nov 2024.

## Builds on (most cited first, 30 of 45)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis](https://mlsanity.com/paper/1910.06711/) (2019)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [WaveGlow: A Flow-based Generative Network for Speech Synthesis](https://mlsanity.com/paper/1811.00002/) (2018)
- [BigVGAN: A Universal Neural Vocoder with Large-Scale Training](https://mlsanity.com/paper/2206.04658/) (2022)
- [Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Data](https://mlsanity.com/paper/2007.08457/) (2020)
- [Do GANs leave artificial fingerprints?](https://mlsanity.com/paper/1812.11842/) (2018)
- [AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks](https://mlsanity.com/paper/2110.01200/) (2021)
- [End-to-end anti-spoofing with RawNet2](https://mlsanity.com/paper/2011.01108/) (2020)
- [ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech](https://mlsanity.com/paper/2207.06389/) (2022)
- [FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis](https://mlsanity.com/paper/2204.09934/) (2022)
- [A Light CNN for Deep Face Representation with Noisy Labels](https://mlsanity.com/paper/1511.02683/) (2015)
- [ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale](https://mlsanity.com/paper/2408.08739/) (2024)
- [WaveFake: A Data Set to Facilitate Audio Deepfake Detection](https://mlsanity.com/paper/2111.02813/) (2021)
- [Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detection](https://mlsanity.com/paper/2101.03778/) (2021)
- [ADD 2023: the Second Audio Deepfake Detection Challenge](https://mlsanity.com/paper/2305.13774/) (2023)
- [Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speech](https://mlsanity.com/paper/2005.05106/) (2020)
- [JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesis](https://mlsanity.com/paper/1711.00354/) (2017)
- [Open-Source Conversational AI with SpeechBrain 1.0](https://mlsanity.com/paper/2407.00463/) (2024)
- [STC Antispoofing Systems for the ASVspoof2019 Challenge](https://mlsanity.com/paper/1904.05576/) (2019)
- [Deep Residual Neural Networks for Audio Spoofing Detection](https://mlsanity.com/paper/1907.00501/) (2019)
- [ASSERT: Anti-Spoofing with Squeeze-Excitation and Residual neTworks](https://mlsanity.com/paper/1904.01120/) (2019)
- [Defending Your Voice: Adversarial Attack on Voice Conversion](https://mlsanity.com/paper/2005.08781/) (2020)
- [Light Convolutional Neural Network with Feature Genuinization for Detection of Synthetic Speech Attacks](https://mlsanity.com/paper/2009.09637/) (2020)
- [Avocodo: Generative Adversarial Network for Artifact-free Vocoder](https://mlsanity.com/paper/2206.13404/) (2022)

## Built on by (most cited first, 3 of 3)

- [Signals of Provenance: Practices & Challenges of Navigating Indicators in AI-Generated Media for Sighted and Blind Individuals](https://mlsanity.com/paper/2505.16057/) (2025)
- [Clean Accuracy Does Not Guarantee Provenance Robustness: A Prospective Codec-Stress Evaluation of Audio Attribution](https://mlsanity.com/paper/2609.07981/) (2026)
- [Tracing Decoder Artifacts for Compact Synthetic Speech Screening](https://mlsanity.com/paper/2609.32050/) (2026)

## Abstract (arXiv)

As speech generation technologies advance, so do risks of impersonation, misinformation, and spoofing. We present a lightweight, training-free approach for detecting synthetic speech and attributing it to its source model. Our method addresses three tasks: (1) single-model attribution in an open-world setting, (2) multi-model attribution in a closed-world setting, and (3) real vs. synthetic speech classification. The core idea is simple: we compute standardized average residuals--the difference between an audio signal and its filtered version--to extract model-agnostic fingerprints that capture synthesis artifacts. Experiments across multiple synthesis systems and languages show AUROC scores above 99%, with strong reliability even when only a subset of model outputs is available. The method maintains high performance under common audio distortions, including echo and moderate background noise, while data augmentation can improve results in more challenging conditions. In addition, out-of-domain detection is performed using Mahalanobis distances to in-domain residual fingerprints, achieving an F1 score of 0.91 on unseen models, reinforcing the method's efficiency, generalizability, and suitability for digital forensics and security applications.