# Mitigating Closed-model Adversarial Examples with Bayesian Neural Modeling for Enhanced End-to-End Speech Recognition

arXiv: https://arxiv.org/abs/2202.08532 · HTML: https://mlsanity.com/paper/2202.08532/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 38 references that are in the map.
- First posted on arXiv 17 Feb 2022.

## Builds on (most cited first, 30 of 30)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models](https://mlsanity.com/paper/1708.03999/) (2017)
- [On Adaptive Attacks to Adversarial Example Defenses](https://mlsanity.com/paper/2002.08347/) (2020)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [Audio Adversarial Examples: Targeted Attacks on Speech-to-Text](https://mlsanity.com/paper/1801.01944/) (2018)
- [Optimal rates for zero-order convex optimization: the power of two function evaluations](https://mlsanity.com/paper/1312.2139/) (2013)
- [Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach](https://mlsanity.com/paper/1801.10578/) (2018)
- [AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networks](https://mlsanity.com/paper/1805.11770/) (2018)
- [Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches](https://mlsanity.com/paper/1803.04386/) (2018)
- [Universal Adversarial Training](https://mlsanity.com/paper/1811.11304/) (2018)
- [GenAttack: Practical Black-box Attacks with Gradient-Free Optimization](https://mlsanity.com/paper/1805.11090/) (2018)
- [Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network](https://mlsanity.com/paper/1810.01279/) (2018)
- [Sliced Wasserstein Kernels for Probability Distributions](https://mlsanity.com/paper/1511.03198/) (2015)
- [Bayesian Layers: A Module for Neural Network Uncertainty](https://mlsanity.com/paper/1812.03973/) (2018)
- [Robust Audio Adversarial Example for a Physical Attack](https://mlsanity.com/paper/1810.11793/) (2018)
- [Characterizing Audio Adversarial Examples Using Temporal Dependency](https://mlsanity.com/paper/1809.10875/) (2018)
- [Multi-task Language Modeling for Improving Speech Recognition of Rare Words](https://mlsanity.com/paper/2011.11715/) (2020)
- [Adversarial Attacks and Defenses for Speech Recognition Systems](https://mlsanity.com/paper/2103.17122/) (2021)
- [Characterizing Speech Adversarial Examples Using Self-Attention U-Net Enhancement](https://mlsanity.com/paper/2003.13917/) (2020)
- [Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning](https://mlsanity.com/paper/2106.00273/) (2021)
- [Comment on "Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network"](https://mlsanity.com/paper/1907.00895/) (2019)
- [Training Augmentation with Adversarial Examples for Robust Speech Recognition](https://mlsanity.com/paper/1806.02782/) (2018)
- [PATE-AAE: Incorporating Adversarial Autoencoder into Private Aggregation of Teacher Ensembles for Spoken Command Classification](https://mlsanity.com/paper/2104.01271/) (2021)
- [Bayesian Adversarial Spheres: Bayesian Inference and Adversarial Examples in a Noiseless Setting](https://mlsanity.com/paper/1811.12335/) (2018)
- [Encrypted Speech Recognition using Deep Polynomial Networks](https://mlsanity.com/paper/1905.05605/) (2019)
- [Adversarial Examples Detection with Bayesian Neural Network](https://mlsanity.com/paper/2105.08620/) (2021)

## Built on by (most cited first, 2 of 2)

- [An Experimental Study on Private Aggregation of Teacher Ensemble Learning for End-to-End Speech Recognition](https://mlsanity.com/paper/2210.05614/) (2022)
- [Investigating Training Strategies and Model Robustness of Low-Rank Adaptation for Language Modeling in Speech Recognition](https://mlsanity.com/paper/2401.10447/) (2024)

## Abstract (arXiv)

In this work, we aim to enhance the system robustness of end-to-end automatic speech recognition (ASR) against adversarially-noisy speech examples. We focus on a rigorous and empirical "closed-model adversarial robustness" setting (e.g., on-device or cloud applications). The adversarial noise is only generated by closed-model optimization (e.g., evolutionary and zeroth-order estimation) without accessing gradient information of a targeted ASR model directly. We propose an advanced Bayesian neural network (BNN) based adversarial detector, which could model latent distributions against adaptive adversarial perturbation with divergence measurement. We further simulate deployment scenarios of RNN Transducer, Conformer, and wav2vec-2.0 based ASR systems with the proposed adversarial detection system. Leveraging the proposed BNN based detection system, we improve detection rate by +2.77 to +5.42% (relative +3.03 to +6.26%) and reduce the word error rate by 5.02 to 7.47% on LibriSpeech datasets compared to the current model enhancement methods against the adversarial speech examples.