# Defending against Adversarial Audio via Diffusion Model

arXiv: https://arxiv.org/abs/2303.01507 · HTML: https://mlsanity.com/paper/2303.01507/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 65 of its 77 references that are in the map.
- First posted on arXiv 2 Mar 2023.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)

## Builds on (most cited first, 30 of 65)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [Square Attack: a query-efficient black-box adversarial attack via random search](https://mlsanity.com/paper/1912.00049/) (2019)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [Adversarial Training for Free!](https://mlsanity.com/paper/1904.12843/) (2019)
- [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020)
- [On Adaptive Attacks to Adversarial Example Defenses](https://mlsanity.com/paper/2002.08347/) (2020)
- [Diffusion Models for Adversarial Purification](https://mlsanity.com/paper/2205.07460/) (2022)
- [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://mlsanity.com/paper/1512.02595/) (2015)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models](https://mlsanity.com/paper/1805.06605/) (2018)

## Built on by (most cited first, 23 of 23)

- [I Can Hear You: Selective Robust Training for Deepfake Audio Detection](https://mlsanity.com/paper/2411.00121/) (2024)
- [MASTERKEY: Practical Backdoor Attack Against Speaker Verification Systems](https://mlsanity.com/paper/2309.06981/) (2023)
- [Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses](https://mlsanity.com/paper/2605.02900/) (2026)
- [Diffusion models meet image counter-forensics](https://mlsanity.com/paper/2311.13629/) (2023)
- [SafeSpeech: Robust and Universal Voice Protection Against Malicious Speech Synthesis](https://mlsanity.com/paper/2504.09839/) (2025)
- [SongBsAb: A Dual Prevention Approach against Singing Voice Conversion based Illegal Song Covers](https://mlsanity.com/paper/2401.17133/) (2024)
- [Prompt2Perturb (P2P): Text-Guided Diffusion-Based Adversarial Attacks on Breast Ultrasound Images](https://mlsanity.com/paper/2412.09910/) (2024)
- [DistriBlock: Identifying adversarial audio samples by leveraging characteristics of the output distribution](https://mlsanity.com/paper/2305.17000/) (2023)
- [Detecting and Defending Against Adversarial Attacks on Automatic Speech Recognition via Diffusion Models](https://mlsanity.com/paper/2409.07936/) (2024)
- [Consistency Purification: Effective and Efficient Diffusion Purification towards Certified Robustness](https://mlsanity.com/paper/2407.00623/) (2024)
- [De-AntiFake: Rethinking the Protective Perturbations Against Voice Cloning Attacks](https://mlsanity.com/paper/2507.02606/) (2025)
- [Demystifying Adversarial Robustness in Diffusion Models: Compression, Randomness, and Geometry](https://mlsanity.com/paper/2505.22839/) (2025)
- [Investigating Vulnerabilities and Defenses Against Audio-Visual Attacks: A Comprehensive Survey Emphasizing Multimodal Models](https://mlsanity.com/paper/2506.11521/) (2025)
- [Adversarial defense based on distribution transfer](https://mlsanity.com/paper/2311.13841/) (2023)
- [Diffusion-C: Unveiling the Generative Challenges of Diffusion Models through Corrupted Data](https://mlsanity.com/paper/2312.08843/) (2023)
- [Are Deep Speech Denoising Models Robust to Adversarial Noise?](https://mlsanity.com/paper/2503.11627/) (2025)
- [VocalBridge: Latent Diffusion-Bridge Purification for Defeating Perturbation-Based Voiceprint Defenses](https://mlsanity.com/paper/2601.02444/) (2026)
- [Precision-Varying Prediction (PVP): Robustifying ASR systems against adversarial attacks](https://mlsanity.com/paper/2603.22590/) (2026)
- [Audio Pirates: Black-box Audio Watermark Removal via Diffusion Priors](https://mlsanity.com/paper/2605.30614/) (2026)
- [Beyond Waveform Robustness: Robust Feature-Vocoder Adversarial Attacks on Automatic Speech Recognition](https://mlsanity.com/paper/2606.05678/) (2026)
- [Psychoacoustically Aligned Latent Smoothing for Adversarial Robustness of Full-Duplex Speech-to-Speech Dialogue Models](https://mlsanity.com/paper/2609.27378/) (2026)
- [LogiC-Diff: Embedding Security Properties Into AI-Enabled Cyber-Physical Systems](https://mlsanity.com/paper/2609.38381/) (2026)
- [From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agents](https://mlsanity.com/paper/2610.15390/) (2026)

## Abstract (arXiv)

Deep learning models have been widely used in commercial acoustic systems in recent years. However, adversarial audio examples can cause abnormal behaviors for those acoustic systems, while being hard for humans to perceive. Various methods, such as transformation-based defenses and adversarial training, have been proposed to protect acoustic systems from adversarial attacks, but they are less effective against adaptive attacks. Furthermore, directly applying the methods from the image domain can lead to suboptimal results because of the unique properties of audio data. In this paper, we propose an adversarial purification-based defense pipeline, AudioPure, for acoustic systems via off-the-shelf diffusion models. Taking advantage of the strong generation ability of diffusion models, AudioPure first adds a small amount of noise to the adversarial audio and then runs the reverse sampling step to purify the noisy audio and recover clean audio. AudioPure is a plug-and-play method that can be directly applied to any pretrained classifier without any fine-tuning or re-training. We conduct extensive experiments on speech command recognition task to evaluate the robustness of AudioPure. Our method is effective against diverse adversarial attacks (e.g. L2 or L_∞-norm). It outperforms the existing methods under both strong adaptive white-box and black-box attacks bounded by L2 or L_∞-norm (up to +20% in robust accuracy). Besides, we also evaluate the certified robustness for perturbations bounded by L2-norm via randomized smoothing. Our pipeline achieves a higher certified accuracy than baselines.