mlsanityData to 14 Oct 2026

Towards the Science of Security and Privacy in Machine Learning

arXiv 1611.03814 ↗First posted 11 Nov 2016cs.CR cs.LGcited by 89

Not on the map. Why?

Abstract (arXiv)

Advances in machine learning (ML) in recent years have enabled a dizzying array of applications such as data analytics, autonomous systems, and security diagnostics. ML is now pervasive---new systems and models are being deployed in every domain imaginable, leading to rapid and widespread deployment of software based inference and decision making. There is growing recognition that ML exposes new vulnerabilities in software systems, yet the technical community's understanding of the nature and extent of these vulnerabilities remains limited. We systematize recent findings on ML security and privacy, focusing on attacks identified on these systems and defenses crafted to date. We articulate a comprehensive threat model for ML, and categorize attacks and defenses within an adversarial framework. Key insights resulting from works both in the ML and security communities are identified and the effectiveness of approaches are related to structural elements of ML algorithms and the data used to train them. We conclude by formally exploring the opposing relationship between model accuracy and resilience to adversarial manipulation. Through these explorations, we show that there are (possibly unavoidable) tensions between model complexity, accuracy, and resilience that must be calibrated for the environments in which they will be used.

Key facts. Cited by 89 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 33; 2022: 21; 2023: 17; 2024: 15; 2025: 3.

89 papers.

Sort:

By area:

  1. 2022Toy Models of Superpositioncited by 971
  2. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  3. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 321
  4. 2021ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Modelscited by 85
  5. 2021Proof-of-Learning: Definitions and Practicecited by 84
  6. 2021Membership Inference Attack on Graph Neural Networkscited by 81
  7. 2021Bad Characters: Imperceptible NLP Attackscited by 71
  8. 2021Probabilistic Margins for Instance Reweighting in Adversarial Trainingcited by 57
  9. 2021Membership Inference Attacks Against Recommender Systemscited by 57
  10. 2021Socially Responsible AI Algorithms: Issues, Purposes, and Challengescited by 56
  11. 2021Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Aheadcited by 41
  12. 2021Quantifying and Mitigating Privacy Risks of Contrastive Learningcited by 36
38 more
  1. 2022Amplifying Membership Exposure via Data Poisoningcited by 34
  2. 2022Federated Learning for Healthcare Domain - Pipeline, Applications and Challengescited by 34
  3. 2022Auditing Membership Leakages of Multi-Exit Networkscited by 27
  4. 2022Architectural Backdoors in Neural Networkscited by 25
  5. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 23
  6. 2023Backdoor Attacks Against Dataset Distillationcited by 21
  7. 2023No Privacy Left Outside: On the (In-)Security of TEE-Shielded DNN Partition for On-Device MLcited by 21
  8. 2021Understanding the Interaction of Adversarial Training with Noisy Labelscited by 19
  9. 2021Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Frameworkcited by 17
  10. 2022Efficient Attribute Unlearning: Towards Selective Removal of Input Attributes from Feature Representationscited by 17
  11. 2022Membership-Doctor: Comprehensive Assessment of Membership Inference Against Machine Learning Modelscited by 16
  12. 2021Robust learning under clean-label attackcited by 15
  13. 2022Adversarial Robustness for Tabular Data through Cost and Utility Awarenesscited by 15
  14. 2021Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfittingcited by 14
  15. 2021Learning and Certification under Instance-targeted Poisoningcited by 13
  16. 2024ElecBench: a Power Dispatch Evaluation Benchmark for Large Language Modelscited by 13
  17. 2023A Survey of Graph Unlearningcited by 12
  18. 2022Improving Adversarial Robustness by Putting More Regularizations on Less Robust Samplescited by 11
  19. 2022DIMBA: Discretely Masked Black-Box Attack in Single Object Trackingcited by 11
  20. 2023Learning from Aggregated Data: Curated Bags versus Random Bagscited by 11
  21. 2024TransLinkGuard: Safeguarding Transformer Models Against Model Stealing in Edge Deploymentcited by 11
  22. 2021Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPScited by 10
  23. 2023Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sourcescited by 10
  24. 2021Towards a Robust and Trustworthy Machine Learning System Development: An Engineering Perspectivecited by 9
  25. 2021Markpainting: Adversarial Machine Learning meets Inpaintingcited by 9
  26. 2021Guided Interpolation for Adversarial Trainingcited by 8
  27. 2021Federated Learning Versus Classical Machine Learning: A Convergence Comparisoncited by 7
  28. 2021The art of defense: letting networks fool the attackercited by 6
  29. 2022Interactive Machine Learning: A State of the Art Reviewcited by 6
  30. 2022Careful What You Wish For: on the Extraction of Adversarially Trained Modelscited by 6
  31. 2023Federated Learning of Shareable Bases for Personalization-Friendly Image Classificationcited by 6
  32. 2024Data Reconstruction Attacks and Defenses: A Systematic Evaluationcited by 6
  33. 2022Robustness Testing of Data and Knowledge Driven Anomaly Detection in Cyber-Physical Systemscited by 5
  34. 2024SpecFormer: Guarding Vision Transformer Robustness via Maximum Singular Value Penalizationcited by 5
  35. 2021Modeling Disclosive Transparency in NLP Application Descriptionscited by 4
  36. 2022Federated Learning with Noisy User Feedbackcited by 4
  37. 2022Never trust, always verify : a roadmap for Trustworthy AI?cited by 4
  38. 2023Can Membership Inferencing be Refuted?cited by 4

50 most cited of 89 All 89 (JSON)