mlsanityData to 7 Oct 2026

Practical Black-Box Attacks against Machine Learning

arXiv 1602.02697 ↗First posted 8 Feb 2016cs.CR cs.LGcited by 893

Not on the map. Why?

Cited by 39 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Machine learning (ML) models, e.g., deep neural networks (DNNs), are vulnerable to adversarial examples: malicious inputs modified to yield erroneous model outputs, while appearing unmodified to human observers. Potential attacks include having malicious content like malware identified as legitimate or controlling vehicle behavior. Yet, all existing adversarial example attacks require knowledge of either the model internals or its training data. We introduce the first practical demonstration of an attacker controlling a remotely hosted DNN with no such knowledge. Indeed, the only capability of our black-box adversary is to observe labels given by the DNN to chosen inputs. Our attack strategy consists in training a local model to substitute for the target DNN, using inputs synthetically generated by an adversary and labeled by the target DNN. We use the local substitute to craft adversarial examples, and find that they are misclassified by the targeted DNN. To perform a real-world and properly-blinded evaluation, we attack a DNN hosted by MetaMind, an online deep learning API. We find that their DNN misclassifies 84.24% of the adversarial examples crafted with our substitute. We demonstrate the general applicability of our strategy to many ML techniques by conducting the same attack against models hosted by Amazon and Google, using logistic regression substitutes. They yield adversarial examples misclassified by Amazon and Google at rates of 96.19% and 88.94%. We also find that this black-box attack strategy is capable of evading defense strategies previously found to make adversarial example crafting harder.

Key facts. Cited by 893 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Feb 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 246; 2022: 196; 2023: 164; 2024: 125; 2025: 95; 2026: 67.

893 papers.

Sort:

By area:

  1. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 307
  2. 2023Low-Resource Languages Jailbreak GPT-4cited by 290
  3. 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 276
  4. 2021Enhancing the Transferability of Adversarial Attacks through Variance Tuningcited by 242
  5. 2023Data-centric Artificial Intelligence: A Surveycited by 202
  6. 2023How Robust is Google's Bard to Adversarial Image Attacks?cited by 177
  7. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  8. 2021Admix: Enhancing the Transferability of Adversarial Attackscited by 155
  9. 2022Frequency Domain Model Augmentation for Adversarial Attackcited by 128
  10. 2021On the Convergence and Robustness of Adversarial Trainingcited by 127
  11. 2022Zero-Shot Machine Unlearningcited by 125
  12. 2021Dataset Inference: Ownership Resolution in Machine Learningcited by 115
38 more
  1. 2021BERT: A Review of Applications in Natural Language Processing and Understandingcited by 109
  2. 2021Adv-Makeup: A New Imperceptible and Transferable Attack on Face Recognitioncited by 95
  3. 2023On the Robustness of Large Multimodal Models Against Image Adversarial Attackscited by 89
  4. 2021Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferabilitycited by 84
  5. 2021Size-Invariant Graph Representations for Graph Classification Extrapolationscited by 82
  6. 2021Proof-of-Learning: Definitions and Practicecited by 81
  7. 2021ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Modelscited by 79
  8. 2022I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defencescited by 75
  9. 2021Manipulating SGD with Data Ordering Attackscited by 73
  10. 2021Bad Characters: Imperceptible NLP Attackscited by 69
  11. 2023Boosting Adversarial Transferability by Block Shuffle and Rotationcited by 64
  12. 2023Black Box Adversarial Prompting for Foundation Modelscited by 62
  13. 2022T-SEA: Transfer-based Self-Ensemble Attack on Object Detectioncited by 59
  14. 2022Fingerprinting Deep Neural Networks Globally via Universal Adversarial Perturbationscited by 58
  15. 2021Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNscited by 57
  16. 2022Towards Data-Free Model Stealing in a Hard Label Settingcited by 57
  17. 2022A Large-scale Multiple-objective Method for Black-box Attack against Object Detectioncited by 55
  18. 2022"Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practicecited by 55
  19. 2021Boosting Adversarial Transferability through Enhanced Momentumcited by 53
  20. 2022MM-BD: Post-Training Detection of Backdoor Attacks with Arbitrary Backdoor Pattern Types Using a Maximum Margin Statisticcited by 53
  21. 2023StyleAdv: Meta Style Adversarial Training for Cross-Domain Few-Shot Learningcited by 53
  22. 2021Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!cited by 51
  23. 2021Adversarial Example Detection for DNN Models: A Review and Experimental Comparisoncited by 51
  24. 2021Model Stealing Attacks Against Inductive Graph Neural Networkscited by 50
  25. 2022Improving the Transferability of Targeted Adversarial Examples through Object-Based Diverse Inputcited by 49
  26. 2021Adversarial Attacks On Multi-Agent Communicationcited by 48
  27. 2023Prompt Stealing Attacks Against Text-to-Image Generation Modelscited by 48
  28. 2024Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Modelscited by 48
  29. 2021Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Drivingcited by 45
  30. 2021Online Adversarial Purification based on Self-Supervisioncited by 45
  31. 2021The 2021 Image Similarity Dataset and Challengecited by 44
  32. 2021Survey: Leakage and Privacy at Inference Timecited by 44
  33. 2021Defending against Model Stealing via Verifying Embedded External Featurescited by 44
  34. 2022A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologiescited by 44
  35. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  36. 2022SSLGuard: A Watermarking Scheme for Self-supervised Learning Pre-trained Encoderscited by 43
  37. 2022Efficient and Effective Augmentation Strategy for Adversarial Trainingcited by 43
  38. 2024A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessmentcited by 43

50 most cited of 893 All 893 (JSON)