# DSVAE: Interpretable Disentangled Representation for Synthetic Speech Detection

arXiv: https://arxiv.org/abs/2304.03323 · HTML: https://mlsanity.com/paper/2304.03323/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 82 references that are in the map.
- First posted on arXiv 6 Apr 2023.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech](https://mlsanity.com/paper/2105.06337/) (2021)
- [Efficient Training of Audio Transformers with Patchout](https://mlsanity.com/paper/2110.05069/) (2021)
- [ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection](https://mlsanity.com/paper/1904.05441/) (2019)
- [Does Audio Deepfake Detection Generalize?](https://mlsanity.com/paper/2203.16263/) (2022)
- [ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild](https://mlsanity.com/paper/2210.02437/) (2022)
- [Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversion](https://mlsanity.com/paper/2008.12527/) (2020)
- [The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods](https://mlsanity.com/paper/1804.04262/) (2018)
- [Face Anti-Spoofing Via Disentangled Representation Learning](https://mlsanity.com/paper/2008.08250/) (2020)
- [On Disentangling Spoof Trace for Generic Face Anti-Spoofing](https://mlsanity.com/paper/2007.09273/) (2020)
- [Learning latent representations for style control and transfer in end-to-end speech synthesis](https://mlsanity.com/paper/1812.04342/) (2018)
- [Theory and Evaluation Metrics for Learning Disentangled Representations](https://mlsanity.com/paper/1908.09961/) (2019)
- [Masked Spectrogram Modeling using Masked Autoencoders for Learning General-purpose Audio Representation](https://mlsanity.com/paper/2204.12260/) (2022)
- [STC Antispoofing Systems for the ASVspoof2019 Challenge](https://mlsanity.com/paper/1904.05576/) (2019)
- [Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesis](https://mlsanity.com/paper/2002.03785/) (2020)
- [Replay and Synthetic Speech Detection with Res2net Architecture](https://mlsanity.com/paper/2010.15006/) (2020)
- [Towards End-to-End Synthetic Speech Detection](https://mlsanity.com/paper/2106.06341/) (2021)
- [Speech is Silver, Silence is Golden: What do ASVspoof-trained Models Really Learn?](https://mlsanity.com/paper/2106.12914/) (2021)
- [MixNMatch: Multifactor Disentanglement and Encoding for Conditional Image Generation](https://mlsanity.com/paper/1911.11758/) (2019)
- [Where and What? Examining Interpretable Disentangled Representations](https://mlsanity.com/paper/2104.05622/) (2021)

## Built on by (most cited first, 6 of 6)

- [DiLA: Disentangled Latent Action World Models](https://mlsanity.com/paper/2605.15725/) (2026)
- [FairSSD: Understanding Bias in Synthetic Speech Detectors](https://mlsanity.com/paper/2404.10989/) (2024)
- [Improving Generalization for AI-Synthesized Voice Detection](https://mlsanity.com/paper/2412.19279/) (2024)
- [VoxGenesis: Unsupervised Discovery of Latent Speaker Manifold for Speech Synthesis](https://mlsanity.com/paper/2403.00529/) (2024)
- [AFSS: Artifact-Focused Self-Synthesis for Mitigating Bias in Audio Deepfake Detection](https://mlsanity.com/paper/2603.26856/) (2026)
- [MF-Speech: Achieving Fine-Grained and Compositional Control in Speech Generation via Factor Disentanglement](https://mlsanity.com/paper/2511.12074/) (2025)

## Abstract (arXiv)

Tools to generate high quality synthetic speech signal that is perceptually indistinguishable from speech recorded from human speakers are easily available. Several approaches have been proposed for detecting synthetic speech. Many of these approaches use deep learning methods as a black box without providing reasoning for the decisions they make. This limits the interpretability of these approaches. In this paper, we propose Disentangled Spectrogram Variational Auto Encoder (DSVAE) which is a two staged trained variational autoencoder that processes spectrograms of speech using disentangled representation learning to generate interpretable representations of a speech signal for detecting synthetic speech. DSVAE also creates an activation map to highlight the spectrogram regions that discriminate synthetic and bona fide human speech signals. We evaluated the representations obtained from DSVAE using the ASVspoof2019 dataset. Our experimental results show high accuracy (>98%) on detecting synthetic speech from 6 known and 10 out of 11 unknown speech synthesizers. We also visualize the representation obtained from DSVAE for 17 different speech synthesizers and verify that they are indeed interpretable and discriminate bona fide and synthetic speech from each of the synthesizers.