# Learning From Yourself: A Self-Distillation Method for Fake Speech Detection

arXiv: https://arxiv.org/abs/2303.01211 · HTML: https://mlsanity.com/paper/2303.01211/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation.
- Builds on 13 of its 19 references that are in the map.
- First posted on arXiv 2 Mar 2023.

## Placed in

- Knowledge distillation / [Speech and audio distillation](https://mlsanity.com/distill/speech-and-audio-distillation/)

## Builds on (most cited first, 13 of 13)

- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation](https://mlsanity.com/paper/1905.08094/) (2019)
- [AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks](https://mlsanity.com/paper/2110.01200/) (2021)
- [End-to-end anti-spoofing with RawNet2](https://mlsanity.com/paper/2011.01108/) (2020)
- [End-to-End Spectro-Temporal Graph Attention Networks for Speaker Verification Anti-Spoofing and Speech Deepfake Detection](https://mlsanity.com/paper/2107.12710/) (2021)
- [One-class Learning Towards Synthetic Voice Spoofing Detection](https://mlsanity.com/paper/2010.13995/) (2020)
- [ASVspoof 2019: spoofing countermeasures for the detection of synthesized, converted and replayed speech](https://mlsanity.com/paper/2102.05889/) (2021)
- [Replay and Synthetic Speech Detection with Res2net Architecture](https://mlsanity.com/paper/2010.15006/) (2020)
- [RawNet: Advanced end-to-end deep neural network using raw waveforms for text-independent speaker verification](https://mlsanity.com/paper/1904.08104/) (2019)
- [Channel-wise Gated Res2Net: Towards Robust Detection of Synthetic Speech Attacks](https://mlsanity.com/paper/2107.08803/) (2021)
- [Self-distillation with Batch Knowledge Ensembling Improves ImageNet Classification](https://mlsanity.com/paper/2104.13298/) (2021)
- [Fully Automated End-to-End Fake Audio Detection](https://mlsanity.com/paper/2208.09618/) (2022)
- [Audio Deepfake Detection Based on a Combination of F0 Information and Real Plus Imaginary Spectrogram Features](https://mlsanity.com/paper/2208.01214/) (2022)

## Built on by (most cited first, 13 of 13)

- [SingFake: Singing Voice Deepfake Detection](https://mlsanity.com/paper/2309.07525/) (2023)
- [Mixture of Experts Fusion for Fake Audio Detection Using Frozen wav2vec 2.0](https://mlsanity.com/paper/2409.11909/) (2024)
- [One-Class Knowledge Distillation for Spoofing Speech Detection](https://mlsanity.com/paper/2309.08285/) (2023)
- [Dual-Branch Knowledge Distillation for Noise-Robust Synthetic Speech Detection](https://mlsanity.com/paper/2310.08869/) (2023)
- [DGSD: Dynamical Graph Self-Distillation for EEG-Based Auditory Spatial Attention Detection](https://mlsanity.com/paper/2309.07147/) (2023)
- [Context-aware TFL: A Universal Context-aware Contrastive Learning Framework for Temporal Forgery Localization](https://mlsanity.com/paper/2506.08493/) (2025)
- [Multi-perspective Information Fusion Res2Net with RandomSpecmix for Fake Speech Detection](https://mlsanity.com/paper/2306.15389/) (2023)
- [Progressive Distillation Based on Masked Generation Feature Method for Knowledge Graph Completion](https://mlsanity.com/paper/2401.12997/) (2024)
- [Frequency-mix Knowledge Distillation for Fake Speech Detection](https://mlsanity.com/paper/2406.09664/) (2024)
- [Are Music Foundation Models Better at Singing Voice Deepfake Detection? Far-Better Fuse them with Speech Foundation Models](https://mlsanity.com/paper/2409.14131/) (2024)
- [EchoFake: A Replay-Aware Dataset for Practical Speech Deepfake Detection](https://mlsanity.com/paper/2510.19414/) (2025)
- [Performance and Complexity Trade-off Optimization of Speech Models During Training](https://mlsanity.com/paper/2601.13704/) (2026)
- [Task-Aware Joint Pruning and Distillation for Efficient Audio Deepfake Detection](https://mlsanity.com/paper/2610.05264/) (2026)

## Abstract (arXiv)

In this paper, we propose a novel self-distillation method for fake speech detection (FSD), which can significantly improve the performance of FSD without increasing the model complexity. For FSD, some fine-grained information is very important, such as spectrogram defects, mute segments, and so on, which are often perceived by shallow networks. However, shallow networks have much noise, which can not capture this very well. To address this problem, we propose using the deepest network instruct shallow network for enhancing shallow networks. Specifically, the networks of FSD are divided into several segments, the deepest network being used as the teacher model, and all shallow networks become multiple student models by adding classifiers. Meanwhile, the distillation path between the deepest network feature and shallow network features is used to reduce the feature difference. A series of experimental results on the ASVspoof 2019 LA and PA datasets show the effectiveness of the proposed method, with significant improvements compared to the baseline.