mlsanityData to 7 Oct 2026

Stealing Machine Learning Models via Prediction APIs

arXiv 1609.02943 ↗First posted 9 Sep 2016cs.CR cs.LG stat.MLcited by 602

Not on the map. Why?

Cited by 87 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Machine learning (ML) models may be deemed confidential due to their sensitive training data, commercial value, or use in security applications. Increasingly often, confidential ML models are being deployed with publicly accessible query interfaces. ML-as-a-service ("predictive analytics") systems are an example: Some allow users to train models on potentially sensitive data and charge others for access on a pay-per-query basis. The tension between model confidentiality and public access motivates our investigation of model extraction attacks. In such attacks, an adversary with black-box access, but no prior knowledge of an ML model's parameters or training data, aims to duplicate the functionality of (i.e., "steal") the model. Unlike in classical learning theory settings, ML-as-a-service offerings may accept partial feature vectors as inputs and include confidence values with predictions. Given these practices, we show simple, efficient attacks that extract target ML models with near-perfect fidelity for popular model classes including logistic regression, neural networks, and decision trees. We demonstrate these attacks against the online services of BigML and Amazon Machine Learning. We further show that the natural countermeasure of omitting confidence values from model outputs still admits potentially harmful model extraction attacks. Our results highlight the need for careful ML model deployment and new model extraction countermeasures.

Key facts. Cited by 602 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Sep 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 109; 2022: 99; 2023: 105; 2024: 99; 2025: 98; 2026: 92.

602 papers.

Sort:

By area:

  1. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
  2. 2023The False Promise of Imitating Proprietary LLMscited by 254
  3. 2021Trustworthy AI: From Principles to Practicescited by 206
  4. 2021Graph Unlearningcited by 129
  5. 2022Reconstructing Training Data from Trained Neural Networkscited by 129
  6. 2021Dataset Inference: Ownership Resolution in Machine Learningcited by 115
  7. 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 113
  8. 2021Trustworthy AI: A Computational Perspectivecited by 93
  9. 2022Membership Inference Attacks by Exploiting Loss Trajectorycited by 93
  10. 2021Membership Inference Attack on Graph Neural Networkscited by 81
  11. 2021Proof-of-Learning: Definitions and Practicecited by 81
  12. 2023Protecting Language Generation Models via Invisible Watermarkingcited by 81
38 more
  1. 2021Node-Level Membership Inference Attacks Against Graph Neural Networkscited by 80
  2. 2021ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Modelscited by 79
  3. 2021Protecting Intellectual Property of Language Generation APIs with Lexical Watermarkcited by 78
  4. 2022I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defencescited by 75
  5. 2021Practical Blind Membership Inference Attack via Differential Comparisonscited by 73
  6. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  7. 2021Software Engineering for AI-Based Systems: A Surveycited by 68
  8. 2021EncoderMI: Membership Inference against Pre-trained Encoders in Contrastive Learningcited by 68
  9. 2021SoK: How Robust is Image Classification Deep Neural Network Watermarking? (Extended Version)cited by 67
  10. 2023Bag of Tricks for Training Data Extraction from Language Modelscited by 67
  11. 2023Language Model Inversioncited by 66
  12. 2021Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecturecited by 63
  13. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  14. 2021Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Imagescited by 58
  15. 2022Fingerprinting Deep Neural Networks Globally via Universal Adversarial Perturbationscited by 58
  16. 2022Towards Data-Free Model Stealing in a Hard Label Settingcited by 57
  17. 2021Socially Responsible AI Algorithms: Issues, Purposes, and Challengescited by 56
  18. 2022"Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practicecited by 55
  19. 2022MM-BD: Post-Training Detection of Backdoor Attacks with Arbitrary Backdoor Pattern Types Using a Maximum Margin Statisticcited by 53
  20. 2021Membership Inference Attacks Against Recommender Systemscited by 52
  21. 2021GraphMI: Extracting Private Graph Data from Graph Neural Networkscited by 50
  22. 2021Model Stealing Attacks Against Inductive Graph Neural Networkscited by 50
  23. 2021Exploiting Explanations for Model Inversion Attackscited by 48
  24. 2022SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learningcited by 47
  25. 2021CryptGPU: Fast Privacy-Preserving Machine Learning on the GPUcited by 46
  26. 2021Survey: Leakage and Privacy at Inference Timecited by 44
  27. 2021Defending against Model Stealing via Verifying Embedded External Featurescited by 44
  28. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  29. 2022SSLGuard: A Watermarking Scheme for Self-supervised Learning Pre-trained Encoderscited by 43
  30. 2021DeepSteal: Advanced Model Extractions Leveraging Efficient Weight Stealing in Memoriescited by 42
  31. 2022Are Your Sensitive Attributes Private? Novel Model Inversion Attribute Inference Attacks on Classification Modelscited by 42
  32. 2022VeriFi: Towards Verifiable Federated Unlearningcited by 41
  33. 2021Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Aheadcited by 40
  34. 2022Structured access: an emerging paradigm for safe AI deploymentcited by 40
  35. 2024Extracting Prompts by Inverting LLM Outputscited by 40
  36. 2023Privacy Side Channels in Machine Learning Systemscited by 38
  37. 2024Logits of API-Protected LLMs Leak Proprietary Informationcited by 38
  38. 2021TnT Attacks! Universal Naturalistic Adversarial Patches Against Deep Neural Network Systemscited by 36

50 most cited of 602 All 602 (JSON)