mlsanityData to 14 Oct 2026

Mental Models of Adversarial Machine Learning

arXiv 2105.03726 ↗First posted 8 May 2021cs.CR cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Although machine learning is widely used in practice, little is known about practitioners' understanding of potential security challenges. In this work, we close this substantial gap and contribute a qualitative study focusing on developers' mental models of the machine learning pipeline and potentially vulnerable components. Similar studies have helped in other security fields to discover root causes or improve risk communication. Our study reveals two facets of practitioners' mental models of machine learning security. Firstly, practitioners often confuse machine learning security with threats and defences that are not directly related to machine learning. Secondly, in contrast to most academic research, our participants perceive security of machine learning as not solely related to individual models, but rather in the context of entire workflows that consist of multiple components. Jointly with our additional findings, these two facets provide a foundation to substantiate mental models for machine learning security and have implications for the integration of adversarial machine learning into corporate workflows, newdecreasing practitioners' reported uncertainty, and appropriate regulatory frameworks for machine learning security.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 90 references that are in the map. First posted on arXiv 8 May 2021.

Builds on

20 of 90 references.

References · 20

  1. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  2. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
  3. 2018Improving fairness in machine learning systems: What do industry practitioners need?cited by 262
  4. 2018Machine Learning with Membership Privacy using Adversarial Regularizationcited by 186
  5. 2020Adversarial Machine Learning -- Industry Perspectivescited by 122
  6. 2021What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Researchcited by 119
  7. 2018Adversarial Reprogramming of Neural Networkscited by 116
  8. 2020Sponge Examples: Energy-Latency Attacks on Neural Networkscited by 111
  9. 2019Bypassing Backdoor Detection Algorithms in Deep Learningcited by 74
  10. 2020How do Data Science Workers Collaborate? Roles, Workflows, and Toolscited by 64
  11. 2021Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needscited by 64
  12. 2018Motivating the Rules of the Game for Adversarial Example Researchcited by 57
8 more
  1. 2018Model-Reuse Attacks on Deep Learning Systemscited by 49
  2. 2019Bad Global Minima Exist and SGD Can Reach Themcited by 49
  3. 2017On the Robustness of Convolutional Neural Networks to Internal Architecture and Weight Perturbationscited by 14
  4. 2020Security and Machine Learning in the Real Worldcited by 9
  5. 2021Understanding Mental Models of AI through Player-AI Interactioncited by 4
  6. 2017Towards Reverse-Engineering Black-Box Neural Networkscited by 2
  7. 2017Why Do Developers Get Password Storage Wrong? A Qualitative Usability Studycited by 1
  8. 2021Surveying the Landscape of Ethics-Focused Design Methodscited by 1

Built on by

Not cited yet.