# Deep Learning and Music Adversaries

arXiv: https://arxiv.org/abs/1507.04761 · HTML: https://mlsanity.com/paper/1507.04761/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2015.

## Built on by (most cited first, 20 of 20)

- [MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training](https://mlsanity.com/paper/2306.00107/) (2023)
- [MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantization](https://mlsanity.com/paper/2501.01108/) (2025)
- [Codified audio language modeling learns useful representations for music information retrieval](https://mlsanity.com/paper/2107.05677/) (2021)
- [MARBLE: Music Audio Representation Benchmark for Universal Evaluation](https://mlsanity.com/paper/2306.10548/) (2023)
- [Low-Resource Music Genre Classification with Cross-Modal Neural Model Reprogramming](https://mlsanity.com/paper/2211.01317/) (2022)
- [Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning](https://mlsanity.com/paper/2106.00273/) (2021)
- [OpenMU: Your Swiss Army Knife for Music Understanding](https://mlsanity.com/paper/2410.15573/) (2024)
- [Who Can Withstand Chat-Audio Attacks? An Evaluation Benchmark for Large Audio-Language Models](https://mlsanity.com/paper/2411.14842/) (2024)
- [Learning Music Sequence Representation from Text Supervision](https://mlsanity.com/paper/2305.19602/) (2023)
- [On the Effectiveness of Speech Self-supervised Learning for Music](https://mlsanity.com/paper/2307.05161/) (2023)
- [CycleDRUMS: Automatic Drum Arrangement For Bass Lines Using CycleGAN](https://mlsanity.com/paper/2104.00353/) (2021)
- [Arabic Synonym BERT-based Adversarial Examples for Text Classification](https://mlsanity.com/paper/2402.03477/) (2024)
- [Myna: Masking-Based Contrastive Learning of Musical Representations](https://mlsanity.com/paper/2502.12511/) (2025)
- [MAIA: An Inpainting-Based Approach for Music Adversarial Attacks](https://mlsanity.com/paper/2509.04980/) (2025)
- [Attacks as Defenses: Designing Robust Audio CAPTCHAs Using Attacks on Automatic Speech Recognition Systems](https://mlsanity.com/paper/2203.05408/) (2022)
- [Audio Contrastive-based Fine-tuning: Decoupling Representation Learning and Classification](https://mlsanity.com/paper/2309.11895/) (2023)
- ["I made this (sort of)": Negotiating authorship, confronting fraudulence, and exploring new musical spaces with prompt-based AI music generation](https://mlsanity.com/paper/2507.23365/) (2025)
- [Are Inherently Interpretable Models More Robust? A Study In Music Emotion Recognition](https://mlsanity.com/paper/2508.03780/) (2025)
- [Linear Complexity Self-Supervised Learning for Music Understanding with Random Quantizer](https://mlsanity.com/paper/2601.09603/) (2026)
- [What Makes a Good Layer? Assessing the Layer-Wise Intrinsic Properties of Music Foundation Models](https://mlsanity.com/paper/2608.14819/) (2026)

## Abstract (arXiv)

An adversary is essentially an algorithm intent on making a classification system perform in some particular way given an input, e.g., increase the probability of a false negative. Recent work builds adversaries for deep learning systems applied to image object recognition, which exploits the parameters of the system to find the minimal perturbation of the input image such that the network misclassifies it with high confidence. We adapt this approach to construct and deploy an adversary of deep learning systems applied to music content analysis. In our case, however, the input to the systems is magnitude spectral frames, which requires special care in order to produce valid input audio signals from network-derived perturbations. For two different train-test partitionings of two benchmark datasets, and two different deep architectures, we find that this adversary is very effective in defeating the resulting systems. We find the convolutional networks are more robust, however, compared with systems based on a majority vote over individually classified audio frames. Furthermore, we integrate the adversary into the training of new deep systems, but do not find that this improves their resilience against the same adversary.