# Addressing Gradient Misalignment in Data-Augmented Training for Robust Speech Deepfake Detection

arXiv: https://arxiv.org/abs/2509.20682 · HTML: https://mlsanity.com/paper/2509.20682/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 33 references that are in the map.
- First posted on arXiv 25 Sep 2025.

## Builds on (most cited first, 27 of 27)

- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Conflict-Averse Gradient Descent for Multi-task Learning](https://mlsanity.com/paper/2110.14048/) (2021)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection](https://mlsanity.com/paper/1904.05441/) (2019)
- [Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models](https://mlsanity.com/paper/2010.05874/) (2020)
- [Does Audio Deepfake Detection Generalize?](https://mlsanity.com/paper/2203.16263/) (2022)
- [ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild](https://mlsanity.com/paper/2210.02437/) (2022)
- [Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation](https://mlsanity.com/paper/2202.12233/) (2022)
- [RawBoost: A Raw Data Boosting and Augmentation Method applied to Automatic Speaker Verification Anti-Spoofing](https://mlsanity.com/paper/2111.04433/) (2021)
- [Domain Generalization via Gradient Surgery](https://mlsanity.com/paper/2108.01621/) (2021)
- [XLSR-Mamba: A Dual-Column Bidirectional State Space Model for Spoofing Attack Detection](https://mlsanity.com/paper/2411.10027/) (2024)
- [Temporal-Channel Modeling in Multi-head Self-Attention for Synthetic Speech Detection](https://mlsanity.com/paper/2406.17376/) (2024)
- [Speech DF Arena: A Leaderboard for Speech DeepFake Detection Models](https://mlsanity.com/paper/2509.02859/) (2025)
- [Mixture of Experts Fusion for Fake Audio Detection Using Frozen wav2vec 2.0](https://mlsanity.com/paper/2409.11909/) (2024)
- [Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-spoofing](https://mlsanity.com/paper/2504.05657/) (2025)
- [Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond](https://mlsanity.com/paper/2501.10945/) (2025)
- [How Do Neural Spoofing Countermeasures Detect Partially Spoofed Audio?](https://mlsanity.com/paper/2406.02483/) (2024)
- [Generalizable Audio Deepfake Detection via Latent Space Refinement and Augmentation](https://mlsanity.com/paper/2501.14240/) (2025)
- [Towards Quantifying and Reducing Language Mismatch Effects in Cross-Lingual Speech Anti-Spoofing](https://mlsanity.com/paper/2409.08346/) (2024)
- [From Sharpness to Better Generalization for Speech Deepfake Detection](https://mlsanity.com/paper/2506.11532/) (2025)
- [Wav2DF-TSL: Two-stage Learning with Efficient Pre-training and Hierarchical Experts Fusion for Robust Audio Deepfake Detection](https://mlsanity.com/paper/2509.04161/) (2025)
- [Bona fide Cross Testing Reveals Weak Spot in Audio Deepfake Detection Systems](https://mlsanity.com/paper/2509.09204/) (2025)
- [Targeted Augmented Data for Audio Deepfake Detection](https://mlsanity.com/paper/2407.07598/) (2024)
- [Room Impulse Responses help attackers to evade Deep Fake Detection](https://mlsanity.com/paper/2409.14712/) (2024)
- [Pushing the Performance of Synthetic Speech Detection with Kolmogorov-Arnold Networks and Self-Supervised Learning Models](https://mlsanity.com/paper/2506.14153/) (2025)

## Built on by (most cited first, 2 of 2)

- [PC-Mix: Partial-Component Audio Spoofing Detection under Mixed Speech and Environmental Sound Conditions](https://mlsanity.com/paper/2607.10345/) (2026)
- [Domain-Adaptive Dual-Gating Mixture of Experts for Generalizable Speech Deepfake Detection](https://mlsanity.com/paper/2609.33709/) (2026)

## Abstract (arXiv)

In speech deepfake detection (SDD), data augmentation (DA) is commonly used to improve model generalization across varied speech conditions and spoofing attacks. However, during training, the backpropagated gradients from original and augmented inputs may misalign, which can result in conflicting parameter updates. These conflicts could hinder convergence and push the model toward suboptimal solutions, thereby reducing the benefits of DA. To investigate and address this issue, we design a dual-path data-augmented (DPDA) training framework with gradient alignment for SDD. In our framework, each training utterance is processed through two input paths: one using the original speech and the other with its augmented version. This design allows us to compare and align their backpropagated gradient directions to reduce optimization conflicts. Our analysis shows that approximately 25% of training iterations exhibit gradient conflicts between the original inputs and their augmented counterparts when using RawBoost augmentation. By resolving these conflicts with gradient alignment, our method accelerates convergence by reducing the number of training epochs and achieves up to an 18.69% relative reduction in Equal Error Rate on the In-the-Wild dataset compared to the baseline.