# On Adversarial Robustness of Large-scale Audio Visual Learning

arXiv: https://arxiv.org/abs/2203.12122 · HTML: https://mlsanity.com/paper/2203.12122/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Mar 2022.

## Built on by (most cited first, 4 of 4)

- [Quantifying and Enhancing Multi-modal Robustness with Modality Preference](https://mlsanity.com/paper/2402.06244/) (2024)
- [Out-of-Distribution Data: An Acquaintance of Adversarial Examples -- A Survey](https://mlsanity.com/paper/2404.05219/) (2024)
- [Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual Active Speaker Detection](https://mlsanity.com/paper/2210.00753/) (2022)
- [Rethinking Audio-Visual Adversarial Vulnerability from Temporal and Modality Perspectives](https://mlsanity.com/paper/2502.11858/) (2025)

## Abstract (arXiv)

As audio-visual systems are being deployed for safety-critical tasks such as surveillance and malicious content filtering, their robustness remains an under-studied area. Existing published work on robustness either does not scale to large-scale dataset, or does not deal with multiple modalities. This work aims to study several key questions related to multi-modal learning through the lens of robustness: 1) Are multi-modal models necessarily more robust than uni-modal models? 2) How to efficiently measure the robustness of multi-modal learning? 3) How to fuse different modalities to achieve a more robust multi-modal model? To understand the robustness of the multi-modal model in a large-scale setting, we propose a density-based metric, and a convexity metric to efficiently measure the distribution of each modality in high-dimensional latent space. Our work provides a theoretical intuition together with empirical evidence showing how multi-modal fusion affects adversarial robustness through these metrics. We further devise a mix-up strategy based on our metrics to improve the robustness of the trained model. Our experiments on AudioSet and Kinetics-Sounds verify our hypothesis that multi-modal models are not necessarily more robust than their uni-modal counterparts in the face of adversarial examples. We also observe our mix-up trained method could achieve as much protection as traditional adversarial training, offering a computationally cheap alternative. Implementation: https://github.com/lijuncheng16/AudioSetDoneRight