Adversarial Machine Learning -- Industry Perspectives
Not on the map. Why?
Cited by 23 in LLM safety and security, but placed by its own abstract, which didn’t fit.
Abstract (arXiv)
Based on interviews with 28 organizations, we found that industry practitioners are not equipped with tactical and strategic tools to protect, detect and respond to attacks on their Machine Learning (ML) systems. We leverage the insights from the interviews and we enumerate the gaps in perspective in securing machine learning systems when viewed in the context of traditional software security development. We write this paper from the perspective of two personas: developers/ML engineers and security incident responders who are tasked with securing ML systems as they are designed, developed and deployed ML systems. The goal of this paper is to engage researchers to revise and amend the Security Development Lifecycle for industrial-grade software in the adversarial ML era.
Key facts. Cited by 121 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Feb 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 16; 2022: 25; 2023: 32; 2024: 24; 2025: 16; 2026: 8.
121 papers.
Sort:
By area:
- 2021Black-box Detection of Backdoor Attacks with Limited Information and Datacited by 71
- 2022A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarkscited by 70
- 2021Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Trainingcited by 58
- 2021Reliable Adversarial Distillation with Unreliable Teacherscited by 55
- 2022"Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practicecited by 55
- 2023TrojanPuzzle: Covertly Poisoning Code-Suggestion Modelscited by 47
- 2021Poison Forensics: Traceback of Data Poisoning Attacks in Neural Networkscited by 37
- 2021PointBA: Towards Backdoor Attacks in 3D Point Cloudcited by 34
- 2022Identifying a Training-Set Attack's Target Using Renormalized Influence Estimationcited by 32
- 2022An Overview of Compressible and Learnable Image Transformation with Secret Key and Its Applicationscited by 29
- 2022Energy-Latency Attacks via Sponge Poisoningcited by 29
- 2024Breaking the False Sense of Security in Backdoor Defense through Re-Activation Attackcited by 26
38 more
- 2022COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attackscited by 25
- 2022Indiscriminate Data Poisoning Attacks on Neural Networkscited by 24
- 2022Machine Learning Security against Data Poisoning: Are We There Yet?cited by 23
- 2023Exploring the Limits of Model-Targeted Indiscriminate Data Poisoning Attackscited by 23
- 2021What Doesn't Kill You Makes You Robust(er): How to Adversarially Train against Data Poisoningcited by 22
- 2023Analyzing And Editing Inner Mechanisms Of Backdoored Language Modelscited by 22
- 2023A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitionerscited by 22
- 2022The Role of Machine Learning in Cybersecuritycited by 21
- 2022Machine Learning Security in Industry: A Quantitative Surveycited by 21
- 2022Friendly Noise against Adversarial Noise: A Powerful Defense against Data Poisoning Attackscited by 20
- 2023Backdoor Attack with Sparse and Invisible Triggercited by 20
- 2022How to Sift Out a Clean Data Subset in the Presence of Data Poisoning?cited by 17
- 2022An Equivalence Between Data Poisoning and Byzantine Gradient Attackscited by 16
- 2023Large Language Models Are Better Adversaries: Exploring Generative Clean-Label Backdoor Attacks Against Text Classifierscited by 14
- 2025DOGe: Defensive Output Generation for LLM Protection Against Knowledge Distillationcited by 14
- 2026The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggerscited by 13
- 2021AI Ethics Principles in Practice: Perspectives of Designers and Developerscited by 12
- 2022Image and Model Transformation with Secret Key for Vision Transformercited by 12
- 2022DeepStability: A Study of Unstable Numerical Methods and Their Solutions in Deep Learningcited by 11
- 2022Holistic Adversarial Robustness of Deep Learning Modelscited by 11
- 2023A Survey on Malware Detection with Graph Representation Learningcited by 11
- 2021Accumulative Poisoning Attacks on Real-time Datacited by 10
- 2021The Hammer and the Nut: Is Bilevel Optimization Really Needed to Poison Linear Classifiers?cited by 9
- 2022Efficient Reward Poisoning Attacks on Online Deep Reinforcement Learningcited by 9
- 2022Threat Assessment in Machine Learning based Systemscited by 9
- 2023SoK: Realistic Adversarial Attacks and Defenses for Intelligent Network Intrusion Detectioncited by 9
- 2023Understanding the robustness difference between stochastic gradient descent and adaptive gradient methodscited by 8
- 2023Bucks for Buckets (B4B): Active Defenses Against Stealing Encoderscited by 8
- 2024Imperio: Language-Guided Backdoor Attacks for Arbitrary Model Controlcited by 8
- 2024Indiscriminate Data Poisoning Attacks on Pre-trained Feature Extractorscited by 8
- 2022Reducing Certified Regression to Certified Classification for General Poisoning Attackscited by 7
- 2022Multi-SpacePhish: Extending the Evasion-space of Adversarial Attacks against Phishing Website Detectors using Machine Learningcited by 7
- 2023Watch Out! Simple Horizontal Class Backdoor Can Trivially Evade Defensecited by 7
- 2021Evaluating the Cybersecurity Risk of Real World, Machine Learning Production Systemscited by 6
- 2023AI Maintenance: A Robustness Perspectivecited by 6
- 2023Large Scale Foundation Models for Intelligent Manufacturing Applications: A Surveycited by 6
- 2024A Method to Facilitate Membership Inference Attacks in Deep Learning Modelscited by 6
- 2024Active Fourier Auditor for Estimating Distributional Properties of ML Modelscited by 6
50 most cited of 121 All 121 (JSON)
- 2026Code-Poisoning Property Inference Attacks17 Jul 2026
- 2026Model Stealing Through the Lens of Model Multiplicity13 Jun 2026
- 2026Focused on the User, Overlooking the Risks: Security and Privacy Understandings, Practices and Challenges of Independent Chinese AI Agent Developers2 Jun 2026
- 2026Are LLM-Enhanced Graph Neural Networks Robust against Poisoning Attacks?27 Mar 2026
- 2026Hybrid Orchestration of Edge AI and Microservices via Graph-based Self-Imitation Learning3 Mar 2026
- 2026Comparative Insights on Adversarial Machine Learning from Industry and Academia: A User-Study Approach4 Feb 2026
- 2026The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggers3 Feb 2026
- 2026The Machine Learning Canvas: Empirical Findings on Why Strategy Matters More Than AI Code Generation5 Jan 2026
- 2025Behavior-Aware and Generalizable Defense Against Black-Box Adversarial Attacks for ML-Based IDS15 Dec 2025
- 2025On Robustness of Linear Classifiers to Targeted Data Poisoning16 Nov 2025
- 2025Are Targeted Data Poisoning Attacks as Effective as We Think?8 Sep 2025
- 2025Defending Against Beta Poisoning Attacks in Machine Learning Models2 Aug 2025
38 more
- 2025Q-Detection: A Quantum-Classical Hybrid Poisoning Attack Detection Method7 Jul 2025
- 2025Vulnerability Disclosure through Adaptive Black-Box Adversarial Attacks on NIDS25 Jun 2025
- 2025Assessing Risk of Stealing Proprietary Models for Medical Imaging Tasks24 Jun 2025
- 2025An Attack Method for Medical Insurance Claim Fraud Detection based on Generative Adversarial Network22 Jun 2025
- 2025Position: Certified Robustness Does Not (Yet) Imply Model Security16 Jun 2025
- 2025Assessing the Resilience of Automotive Intrusion Detection Systems to Adversarial Manipulation12 Jun 2025
- 2025DOGe: Defensive Output Generation for LLM Protection Against Knowledge Distillation26 May 2025
- 2025Cert-SSBD: Certified Backdoor Defense with Sample-Specific Smoothing Noises30 Apr 2025
- 2025The Ultimate Cookbook for Invisible Poison: Crafting Subtle Clean-Label Text Backdoors with Style Attributes24 Apr 2025
- 2025MMARD: Improving the Min-Max Optimization Process in Adversarial Robustness Distillation9 Mar 2025
- 2025AnywhereDoor: Multi-Target Backdoor Attacks on Object Detection9 Mar 2025
- 2025Adversarial Training for Defense Against Label Poisoning Attacks24 Feb 2025
- 2024V2-SfMLearner: Learning Monocular Depth and Ego-motion for Multimodal Wireless Capsule Endoscopy23 Dec 2024
- 2024Exact Certification of (Graph) Neural Networks Against Label Poisoning30 Nov 2024
- 2024Adversarial Data Poisoning Attacks on Quantum Machine Learning in the NISQ Era21 Nov 2024
- 2024AnywhereDoor: Multi-Target Backdoor Attacks on Object Detection21 Nov 2024
- 2024Taking off the Rose-Tinted Glasses: A Critical Look at Adversarial ML Through the Lens of Evasion Attacks15 Oct 2024
- 2024Adversarially Guided Stateful Defense Against Backdoor Attacks in Federated Deep Learning15 Oct 2024
- 2024Uncovering, Explaining, and Mitigating the Superficial Safety of Backdoor Defense13 Oct 2024
- 2024Fragile Giants: Understanding the Susceptibility of Models to Subpopulation Attacks11 Oct 2024
- 2024Active Fourier Auditor for Estimating Distributional Properties of ML Models10 Oct 2024
- 2024The poison of dimensionality25 Sep 2024
- 2024Flatness-aware Sequential Learning Generates Resilient Backdoors20 Jul 2024
- 2024Provable Robustness of (Graph) Neural Networks Against Data Poisoning and Backdoor Attacks15 Jul 2024
- 2024Model-agnostic clean-label backdoor mitigation in cybersecurity environments11 Jul 2024
- 2024Mitigating Backdoor Attacks using Activation-Guided Model Editing10 Jul 2024
- 2024A Method to Facilitate Membership Inference Attacks in Deep Learning Models2 Jul 2024
- 2024SecGenAI: Enhancing Security of Cloud-based Generative AI Applications within Australian Critical Technologies of National Interest1 Jul 2024
- 2024Adversaries With Incentives: A Strategic Alternative to Adversarial Robustness17 Jun 2024
- 2024Adversarial Evasion Attack Efficiency against Large Language Models12 Jun 2024
- 2024Breaking the False Sense of Security in Backdoor Defense through Re-Activation Attack25 May 2024
- 2024Mitigating Label Flipping Attacks in Malicious URL Detectors Using Ensemble Trees5 Mar 2024
- 2024Indiscriminate Data Poisoning Attacks on Pre-trained Feature Extractors20 Feb 2024
- 2024Enhancing the Fairness and Performance of Edge Cameras with Explainable AI18 Jan 2024
- 2024ML-On-Rails: Safeguarding Machine Learning Models in Software Systems A Case Study12 Jan 2024
- 2024Imperio: Language-Guided Backdoor Attacks for Arbitrary Model Control2 Jan 2024
- 2023Attack Tree Analysis for Adversarial Evasion Attacks28 Dec 2023
- 2023Manipulating Trajectory Prediction with Backdoors21 Dec 2023
50 newest of 121 All 121 (JSON)
- 2022A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarkscited by 70
- 2021Poison Forensics: Traceback of Data Poisoning Attacks in Neural Networkscited by 37
- 2022Indiscriminate Data Poisoning Attacks on Neural Networkscited by 24
- 2022Machine Learning Security against Data Poisoning: Are We There Yet?cited by 23
- 2023Exploring the Limits of Model-Targeted Indiscriminate Data Poisoning Attackscited by 23
- 2021What Doesn't Kill You Makes You Robust(er): How to Adversarially Train against Data Poisoningcited by 22
- 2023Analyzing And Editing Inner Mechanisms Of Backdoored Language Modelscited by 22
- 2022Friendly Noise against Adversarial Noise: A Powerful Defense against Data Poisoning Attackscited by 20
- 2022How to Sift Out a Clean Data Subset in the Presence of Data Poisoning?cited by 17
- 2022An Equivalence Between Data Poisoning and Byzantine Gradient Attackscited by 16
- 2023Large Language Models Are Better Adversaries: Exploring Generative Clean-Label Backdoor Attacks Against Text Classifierscited by 14
- 2026The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggerscited by 13
12 most cited of 23 All 23 (JSON)
- 2026The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggers3 Feb 2026
- 2025On Robustness of Linear Classifiers to Targeted Data Poisoning16 Nov 2025
- 2025Are Targeted Data Poisoning Attacks as Effective as We Think?8 Sep 2025
- 2025The Ultimate Cookbook for Invisible Poison: Crafting Subtle Clean-Label Text Backdoors with Style Attributes24 Apr 2025
- 2024Adversarial Data Poisoning Attacks on Quantum Machine Learning in the NISQ Era21 Nov 2024
- 2024Flatness-aware Sequential Learning Generates Resilient Backdoors20 Jul 2024
- 2024A Method to Facilitate Membership Inference Attacks in Deep Learning Models2 Jul 2024
- 2024Mitigating Label Flipping Attacks in Malicious URL Detectors Using Ensemble Trees5 Mar 2024
- 2024Indiscriminate Data Poisoning Attacks on Pre-trained Feature Extractors20 Feb 2024
- 2024Imperio: Language-Guided Backdoor Attacks for Arbitrary Model Control2 Jan 2024
- 2023Large Language Models Are Better Adversaries: Exploring Generative Clean-Label Backdoor Attacks Against Text Classifiers28 Oct 2023
- 2023Pick your Poison: Undetectability versus Robustness in Data Poisoning Attacks7 May 2023
12 newest of 23 All 23 (JSON)
- 2021Black-box Detection of Backdoor Attacks with Limited Information and Datacited by 71
- 2021Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Trainingcited by 58
- 2021Reliable Adversarial Distillation with Unreliable Teacherscited by 55
- 2022"Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practicecited by 55
- 2023TrojanPuzzle: Covertly Poisoning Code-Suggestion Modelscited by 47
- 2021PointBA: Towards Backdoor Attacks in 3D Point Cloudcited by 34
- 2022Identifying a Training-Set Attack's Target Using Renormalized Influence Estimationcited by 32
- 2022An Overview of Compressible and Learnable Image Transformation with Secret Key and Its Applicationscited by 29
- 2022Energy-Latency Attacks via Sponge Poisoningcited by 29
- 2024Breaking the False Sense of Security in Backdoor Defense through Re-Activation Attackcited by 26
- 2023A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitionerscited by 22
- 2022The Role of Machine Learning in Cybersecuritycited by 21
12 most cited of 95 All 95 (JSON)
- 2026Code-Poisoning Property Inference Attacks17 Jul 2026
- 2026Model Stealing Through the Lens of Model Multiplicity13 Jun 2026
- 2026Are LLM-Enhanced Graph Neural Networks Robust against Poisoning Attacks?27 Mar 2026
- 2026Comparative Insights on Adversarial Machine Learning from Industry and Academia: A User-Study Approach4 Feb 2026
- 2026The Machine Learning Canvas: Empirical Findings on Why Strategy Matters More Than AI Code Generation5 Jan 2026
- 2025Behavior-Aware and Generalizable Defense Against Black-Box Adversarial Attacks for ML-Based IDS15 Dec 2025
- 2025Defending Against Beta Poisoning Attacks in Machine Learning Models2 Aug 2025
- 2025Q-Detection: A Quantum-Classical Hybrid Poisoning Attack Detection Method7 Jul 2025
- 2025Vulnerability Disclosure through Adaptive Black-Box Adversarial Attacks on NIDS25 Jun 2025
- 2025Assessing Risk of Stealing Proprietary Models for Medical Imaging Tasks24 Jun 2025
- 2025An Attack Method for Medical Insurance Claim Fraud Detection based on Generative Adversarial Network22 Jun 2025
- 2025Position: Certified Robustness Does Not (Yet) Imply Model Security16 Jun 2025
12 newest of 95 All 95 (JSON)