# Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-spoofing

arXiv: https://arxiv.org/abs/2504.05657 · HTML: https://mlsanity.com/paper/2504.05657/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 65 of its 85 references that are in the map.
- First posted on arXiv 8 Apr 2025.

## Builds on (most cited first, 30 of 65)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech](https://mlsanity.com/paper/2007.06028/) (2020)
- [AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks](https://mlsanity.com/paper/2110.01200/) (2021)
- [ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection](https://mlsanity.com/paper/2109.00537/) (2021)
- [Does Audio Deepfake Detection Generalize?](https://mlsanity.com/paper/2203.16263/) (2022)
- [Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation](https://mlsanity.com/paper/2202.12233/) (2022)
- [Attentive Statistics Pooling for Deep Speaker Embedding](https://mlsanity.com/paper/1803.10963/) (2018)
- [ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale](https://mlsanity.com/paper/2408.08739/) (2024)
- [RawBoost: A Raw Data Boosting and Augmentation Method applied to Automatic Speaker Verification Anti-Spoofing](https://mlsanity.com/paper/2111.04433/) (2021)
- [ADD 2023: the Second Audio Deepfake Detection Challenge](https://mlsanity.com/paper/2305.13774/) (2023)
- [A Comparative Study on Recent Neural Spoofing Countermeasures for Synthetic Speech Detection](https://mlsanity.com/paper/2103.11326/) (2021)
- [MLAAD: The Multi-Language Audio Anti-Spoofing Dataset](https://mlsanity.com/paper/2401.09512/) (2024)
- [The PartialSpoof Database and Countermeasures for the Detection of Short Fake Speech Segments Embedded in an Utterance](https://mlsanity.com/paper/2204.05177/) (2022)
- [Investigating self-supervised front ends for speech spoofing countermeasures](https://mlsanity.com/paper/2111.07725/) (2021)
- [An Enhanced Res2Net with Local and Global Feature Fusion for Speaker Verification](https://mlsanity.com/paper/2305.12838/) (2023)
- [The Vicomtech Audio Deepfake Detection System based on Wav2Vec2 for the 2022 ADD Challenge](https://mlsanity.com/paper/2203.01573/) (2022)
- [The Codecfake Dataset and Countermeasures for the Universally Detection of Deepfake Audio](https://mlsanity.com/paper/2405.04880/) (2024)
- [XLSR-Mamba: A Dual-Column Bidirectional State Space Model for Spoofing Attack Detection](https://mlsanity.com/paper/2411.10027/) (2024)

## Built on by (most cited first, 19 of 19)

- [Speech DF Arena: A Leaderboard for Speech DeepFake Detection Models](https://mlsanity.com/paper/2509.02859/) (2025)
- [Multilingual Source Tracing of Speech Deepfakes: A First Benchmark](https://mlsanity.com/paper/2508.04143/) (2025)
- [TRACE: Training-Free Partial Audio Deepfake Detection via Embedding Trajectory Analysis of Speech Foundation Models](https://mlsanity.com/paper/2604.01083/) (2026)
- [EmoTra-TTS: Smooth Intra-Utterance Emotion Transitions for Speech Synthesis](https://mlsanity.com/paper/2608.23791/) (2026)
- [Pushing the Performance of Synthetic Speech Detection with Kolmogorov-Arnold Networks and Self-Supervised Learning Models](https://mlsanity.com/paper/2506.14153/) (2025)
- [Addressing Gradient Misalignment in Data-Augmented Training for Robust Speech Deepfake Detection](https://mlsanity.com/paper/2509.20682/) (2025)
- [The Impact of Audio Watermarking on Audio Anti-Spoofing Countermeasures](https://mlsanity.com/paper/2509.20736/) (2025)
- [QAMO: Quality-aware Multi-centroid One-class Learning For Speech Deepfake Detection](https://mlsanity.com/paper/2509.20679/) (2025)
- [Generalizable Speech Deepfake Detection via Information Bottleneck Enhanced Adversarial Alignment](https://mlsanity.com/paper/2509.23618/) (2025)
- [On Deepfake Voice Detection -- It's All in the Presentation](https://mlsanity.com/paper/2509.26471/) (2025)
- [Teffic-Audio: Tell Fact from Fiction](https://mlsanity.com/paper/2607.28351/) (2026)
- [Acoustic scattering AI for non-invasive object classifications: A case study on hair assessment](https://mlsanity.com/paper/2506.14148/) (2025)
- [Dual-Granularity Orthogonal Disentanglement for Generalizable Audio Deepfake Detection](https://mlsanity.com/paper/2606.16532/) (2026)
- [Supervised Post-training of Speech Foundation Models for Robust Adaptation in Speech Deepfake Detection](https://mlsanity.com/paper/2606.25328/) (2026)
- [Time-Frequency Consistency Learning for Robust Speech Deepfake Detection](https://mlsanity.com/paper/2607.17761/) (2026)
- [What Survives the Codec Shift: Pooled No-Vocals Residuals for Speech Deepfake Detection](https://mlsanity.com/paper/2609.33375/) (2026)
- [Domain-Adaptive Dual-Gating Mixture of Experts for Generalizable Speech Deepfake Detection](https://mlsanity.com/paper/2609.33709/) (2026)
- [Adaptive Compression and Targeted Perturbation: A Unified Framework for Generalized Audio Deepfake Detection](https://mlsanity.com/paper/2610.16049/) (2026)
- [CAFE: Confining Feature Distillation to Language-Routed Adapters for Compact Multilingual Automatic Speech Recognition](https://mlsanity.com/paper/2610.19134/) (2026)

## Abstract (arXiv)

Speech foundation models have significantly advanced various speech-related tasks by providing exceptional representation capabilities. However, their high-dimensional output features often create a mismatch with downstream task models, which typically require lower-dimensional inputs. A common solution is to apply a dimensionality reduction (DR) layer, but this approach increases parameter overhead, computational costs, and risks losing valuable information. To address these issues, we propose Nested Res2Net (Nes2Net), a lightweight back-end architecture designed to directly process high-dimensional features without DR layers. The nested structure enhances multi-scale feature extraction, improves feature interaction, and preserves high-dimensional information. We first validate Nes2Net on CtrSVDD, a singing voice deepfake detection dataset, and report a 22% performance improvement and an 87% back-end computational cost reduction over the state-of-the-art baseline. Additionally, extensive testing across four diverse datasets: ASVspoof 2021, ASVspoof 5, PartialSpoof, and In-the-Wild, covering fully spoofed speech, adversarial attacks, partial spoofing, and real-world scenarios, consistently highlights Nes2Net's superior robustness and generalization capabilities. The code package and pre-trained models are available at https://github.com/Liu-Tianchi/Nes2Net.