mlsanityData to 7 Oct 2026

Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples

arXiv 1605.07277 ↗First posted 24 May 2016cs.CR cs.LGcited by 439

Not on the map. Why?

Cited by 28 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Many machine learning models are vulnerable to adversarial examples: inputs that are specially crafted to cause a machine learning model to produce an incorrect output. Adversarial examples that affect one model often affect another model, even if the two models have different architectures or were trained on different training sets, so long as both models were trained to perform the same task. An attacker may therefore train their own substitute model, craft adversarial examples against the substitute, and transfer them to a victim model, with very little information about the victim. Recent work has further developed a technique that uses the victim model as an oracle to label a synthetic training set for the substitute, so the attacker need not even collect a training set to mount the attack. We extend these recent techniques using reservoir sampling to greatly enhance the efficiency of the training procedure for the substitute model. We introduce new transferability attacks between previously unexplored (substitute, victim) pairs of machine learning model classes, most notably SVMs and decision trees. We demonstrate our attacks on two commercial machine learning classification systems from Amazon (96.19% misclassification rate) and Google (88.94%) using only 800 queries of the victim model, thereby showing that existing machine learning approaches are in general vulnerable to systematic black-box attacks regardless of their structure.

Key facts. Cited by 439 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 May 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 121; 2022: 97; 2023: 78; 2024: 68; 2025: 53; 2026: 22.

439 papers.

Sort:

By area:

  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  2. 2022Red Teaming Language Models with Language Modelscited by 983
  3. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  4. 2023Detecting Language Model Attacks with Perplexitycited by 411
  5. 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 281
  6. 2023Glaze: Protecting Artists from Style Mimicry by Text-to-Image Modelscited by 200
  7. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  8. 2023Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examplescited by 163
  9. 2023Raising the Cost of Malicious AI-Powered Image Editingcited by 160
  10. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 153
  11. 2023Diffusion Models for Imperceptible and Transferable Adversarial Attackcited by 93
  12. 2023On the Robustness of Large Multimodal Models Against Image Adversarial Attackscited by 89
38 more
  1. 2022I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defencescited by 75
  2. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  3. 2021Person Re-identification Method Based on Color Attack and Joint Defencecited by 56
  4. 2024Query-Based Adversarial Prompt Generationcited by 55
  5. 2021Adversarial Example Detection for DNN Models: A Review and Experimental Comparisoncited by 51
  6. 2022Improving the Transferability of Targeted Adversarial Examples through Object-Based Diverse Inputcited by 49
  7. 2021Adversarial Attacks for Tabular Data: Application to Fraud Detection and Imbalanced Datacited by 45
  8. 2021Data Poisoning Won't Save You From Facial Recognitioncited by 45
  9. 2022A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologiescited by 44
  10. 2021Optical Adversarial Attackcited by 43
  11. 2022Pruning has a disparate impact on model accuracycited by 43
  12. 2023A Survey on Transferability of Adversarial Examples across Deep Neural Networkscited by 43
  13. 2021DeepSteal: Advanced Model Extractions Leveraging Efficient Weight Stealing in Memoriescited by 42
  14. 2022Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domainscited by 42
  15. 2025We're Different, We're the Same: Creative Homogeneity Across LLMscited by 41
  16. 2021On the Certified Robustness for Ensemble Models and Beyondcited by 40
  17. 2024An Image Is Worth 1000 Lies: Adversarial Transferability across Prompts on Vision-Language Modelscited by 40
  18. 2022LGV: Boosting Adversarial Example Transferability from Large Geometric Vicinitycited by 39
  19. 2022Query-Efficient Black-box Adversarial Attacks Guided by a Transfer-based Priorcited by 38
  20. 2021TRS: Transferability Reduced Ensemble via Encouraging Gradient Diversity and Model Smoothnesscited by 37
  21. 2023Did You Train on My Dataset? Towards Public Dataset Protection with Clean-Label Backdoor Watermarkingcited by 37
  22. 2023Understanding the Robustness of 3D Object Detection with Bird's-Eye-View Representations in Autonomous Drivingcited by 37
  23. 2021TnT Attacks! Universal Naturalistic Adversarial Patches Against Deep Neural Network Systemscited by 36
  24. 2023OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimizationcited by 36
  25. 2021Physical Adversarial Attacks on an Aerial Imagery Object Detectorcited by 35
  26. 2021Trivial or impossible -- dichotomous data difficulty masks model differences (on ImageNet and beyond)cited by 35
  27. 2022Transferable Adversarial Attack based on Integrated Gradientscited by 35
  28. 2021GradDiv: Adversarial Robustness of Randomized Neural Networks via Gradient Diversity Regularizationcited by 34
  29. 2021A Little Robustness Goes a Long Way: Leveraging Robust Features for Targeted Transfer Attackscited by 33
  30. 2024Resurrecting Old Classes with New Data for Exemplar-Free Continual Learningcited by 33
  31. 2023Making Substitute Models More Bayesian Can Enhance Transferability of Adversarial Examplescited by 32
  32. 2025Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attackscited by 32
  33. 2024Universal Adversarial Perturbations for Vision-Language Pre-trained Modelscited by 31
  34. 2023Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectivescited by 30
  35. 2021Low Curvature Activations Reduce Overfitting in Adversarial Trainingcited by 29
  36. 2023Generalizable Black-Box Adversarial Attack with Meta Learningcited by 29
  37. 2023Improving Adversarial Transferability via Intermediate-level Perturbation Decaycited by 29
  38. 2023ChatGPT as an Attack Tool: Stealthy Textual Backdoor Attack via Blackbox Generative Model Triggercited by 29

50 most cited of 439 All 439 (JSON)