# Machine Learning Security in Industry: A Quantitative Survey

arXiv: https://arxiv.org/abs/2207.05164 · HTML: https://mlsanity.com/paper/2207.05164/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 47 references that are in the map.
- First posted on arXiv 11 Jul 2022.

## Builds on (most cited first, 16 of 16)

- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [On Adaptive Attacks to Adversarial Example Defenses](https://mlsanity.com/paper/2002.08347/) (2020)
- [Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples](https://mlsanity.com/paper/1605.07277/) (2016)
- [Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning](https://mlsanity.com/paper/1712.03141/) (2017)
- [Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models](https://mlsanity.com/paper/1708.08296/) (2017)
- [Machine Learning with Membership Privacy using Adversarial Regularization](https://mlsanity.com/paper/1807.05852/) (2018)
- [PRADA: Protecting against DNN Model Stealing Attacks](https://mlsanity.com/paper/1805.02628/) (2018)
- [Adversarial Machine Learning -- Industry Perspectives](https://mlsanity.com/paper/2002.05646/) (2020)
- [Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning](https://mlsanity.com/paper/2205.01992/) (2022)
- [Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs](https://mlsanity.com/paper/2101.09824/) (2021)
- [Motivating the Rules of the Game for Adversarial Example Research](https://mlsanity.com/paper/1807.06732/) (2018)
- [The Threat of Offensive AI to Organizations](https://mlsanity.com/paper/2106.15764/) (2021)

## Built on by (most cited first, 21 of 21)

- [Backdoor Attack with Sparse and Invisible Trigger](https://mlsanity.com/paper/2306.06209/) (2023)
- [Towards more Practical Threat Models in Artificial Intelligence Security](https://mlsanity.com/paper/2311.09994/) (2023)
- [Backdoor Secrets Unveiled: Identifying Backdoor Data with Optimized Scaled Prediction Consistency](https://mlsanity.com/paper/2403.10717/) (2024)
- ["X of Information'' Continuum: A Survey on AI-Driven Multi-dimensional Metrics for Next-Generation Networked Systems](https://mlsanity.com/paper/2507.19657/) (2025)
- [Adversarial Machine Learning for Social Good: Reframing the Adversary as an Ally](https://mlsanity.com/paper/2310.03614/) (2023)
- [Exact Certification of (Graph) Neural Networks Against Label Poisoning](https://mlsanity.com/paper/2412.00537/) (2024)
- [Provable Robustness of (Graph) Neural Networks Against Data Poisoning and Backdoor Attacks](https://mlsanity.com/paper/2407.10867/) (2024)
- [Efficient and Effective Model Extraction](https://mlsanity.com/paper/2409.14122/) (2024)
- [Poisoning Network Flow Classifiers](https://mlsanity.com/paper/2306.01655/) (2023)
- [A Red Teaming Framework for Securing AI in Maritime Autonomous Systems](https://mlsanity.com/paper/2312.11500/) (2023)
- [Adversarial Data Poisoning Attacks on Quantum Machine Learning in the NISQ Era](https://mlsanity.com/paper/2411.14412/) (2024)
- [The Strong, Weak and Benign Goodhart's law. An independence-free and paradigm-agnostic formalisation](https://mlsanity.com/paper/2505.23445/) (2025)
- [TRACE-Bot: Detecting Emerging LLM-Driven Social Bots via Implicit Semantic Representations and AIGC-Enhanced Behavioral Patterns](https://mlsanity.com/paper/2604.02147/) (2026)
- [Transferable Adversarial Robustness for Categorical Data via Universal Robust Embeddings](https://mlsanity.com/paper/2306.04064/) (2023)
- [Footprints of Data in a Classifier: Understanding the Privacy Risks and Solution Strategies](https://mlsanity.com/paper/2407.02268/) (2024)
- [Graph of Effort: Quantifying Risk of AI Usage for Vulnerability Assessment](https://mlsanity.com/paper/2503.16392/) (2025)
- [Measuring Equality in Machine Learning Security Defenses: A Case Study in Speech Recognition](https://mlsanity.com/paper/2302.08973/) (2023)
- [MLSMM: Machine Learning Security Maturity Model](https://mlsanity.com/paper/2306.16127/) (2023)
- [Preventing Adversarial AI Attacks Against Autonomous Situational Awareness: A Maritime Case Study](https://mlsanity.com/paper/2505.21609/) (2025)
- [Expanding ML-Documentation Standards For Better Security](https://mlsanity.com/paper/2507.12003/) (2025)
- [I Stolenly Swear That I Am Up to (No) Good: Design and Evaluation of Model Stealing Attacks](https://mlsanity.com/paper/2508.21654/) (2025)

## Abstract (arXiv)

Despite the large body of academic work on machine learning security, little is known about the occurrence of attacks on machine learning systems in the wild. In this paper, we report on a quantitative study with 139 industrial practitioners. We analyze attack occurrence and concern and evaluate statistical hypotheses on factors influencing threat perception and exposure. Our results shed light on real-world attacks on deployed machine learning. On the organizational level, while we find no predictors for threat exposure in our sample, the amount of implement defenses depends on exposure to threats or expected likelihood to become a target. We also provide a detailed analysis of practitioners' replies on the relevance of individual machine learning attacks, unveiling complex concerns like unreliable decision making, business information leakage, and bias introduction into models. Finally, we find that on the individual level, prior knowledge about machine learning security influences threat perception. Our work paves the way for more research about adversarial machine learning in practice, but yields also insights for regulation and auditing.