mlsanityData to 7 Oct 2026

SoK: Colluding Adversaries in Machine Learning Pipelines

arXiv 2606.10091 ↗First posted 8 Jun 2026cs.CR cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Machine learning (ML) models are susceptible to various security, privacy, and fairness risks. Adversaries with different characteristics (i.e., objectives, knowledge, and capabilities) can collude by executing one attack to amplify others. Existing work lacks a systematic framework to explore collusion among adversaries, and to study the implications of the adversaries' characteristics. We present a framework covering collusion (a) between train- and inference-time adversaries, and (b) among inference-time adversaries. Our framework accounts for factors enabling collusion between adversaries. We propose a guideline to conjecture about the potential for collusion using enabling factors. We use it to explain prior work, conjecture about unexplored collusions, and empirically validate five such cases. Finally, we discuss how adversaries' characteristics influence the potential for collusion.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 99 of its 131 references that are in the map. First posted on arXiv 8 Jun 2026.

Builds on

99 of 131 references.

References · 99

  1. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  2. 2020Extracting Training Data from Large Language Modelscited by 2,144
  3. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  4. 2019Deep Leakage from Gradientscited by 1,004
  5. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  6. 2021Membership Inference Attacks From First Principlescited by 839
  7. 2023Extracting Training Data from Diffusion Modelscited by 723
  8. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  9. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
  10. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  11. 2016Stealing Machine Learning Models via Prediction APIscited by 602
  12. 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
38 more
  1. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
  2. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  3. 2020Backdoor Learning: A Surveycited by 412
  4. 2020Label-Only Membership Inference Attackscited by 371
  5. 2018Knockoff Nets: Stealing Functionality of Black-Box Modelscited by 285
  6. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  7. 2019The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networkscited by 241
  8. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 225
  9. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  10. 2024Do Membership Inference Attacks Work on Large Language Models?cited by 188
  11. 2020Blind Backdoors in Deep Learning Modelscited by 174
  12. 2017Machine Learning Models that Remember Too Muchcited by 165
  13. 2018PRADA: Protecting against DNN Model Stealing Attackscited by 159
  14. 2024Stealing Part of a Production Language Modelcited by 152
  15. 2020A Survey of Privacy Attacks in Machine Learningcited by 143
  16. 2019Thieves on Sesame Street! Model Extraction of BERT-based APIscited by 136
  17. 2023Are Diffusion Models Vulnerable to Membership Inference Attacks?cited by 135
  18. 2022The Privacy Onion Effect: Memorization is Relativecited by 128
  19. 2018Stealing Hyperparameters in Machine Learningcited by 115
  20. 2021Adversarial Examples Make Strong Poisonscited by 115
  21. 2020Sponge Examples: Energy-Latency Attacks on Neural Networkscited by 108
  22. 2022Truth Serum: Poisoning Machine Learning Models to Reveal Their Secretscited by 94
  23. 2022Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoningcited by 93
  24. 2023Circumventing Concept Erasure Methods For Text-to-Image Generative Modelscited by 81
  25. 2022I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defencescited by 75
  26. 2021Manipulating SGD with Data Ordering Attackscited by 73
  27. 2020Poisoning Attacks on Algorithmic Fairnesscited by 58
  28. 2018Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architecturescited by 56
  29. 2021Property Inference From Poisoningcited by 56
  30. 2020DeepHammer: Depleting the Intelligence of Deep Neural Networks through Targeted Chain of Bit Flipscited by 53
  31. 2023Leveraging Optimization for Adaptive Attacks on Image Watermarkscited by 50
  32. 2022Are Attribute Inference Attacks Just Imputation?cited by 48
  33. 2020Exacerbating Algorithmic Bias through Fairness Attackscited by 47
  34. 2022Differential Privacy and Fairness in Decisions and Learning Tasks: A Surveycited by 47
  35. 2022SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learningcited by 47
  36. 2020Leakage of Dataset Properties in Multi-Party Machine Learningcited by 46
  37. 2023GIFD: A Generative Gradient Inversion Method with Feature Domain Optimizationcited by 45
  38. 2023A Survey on Transferability of Adversarial Examples across Deep Neural Networkscited by 43

50 most cited of 99 All 99 (JSON)

Built on by

Not cited yet.