mlsanityData to 7 Oct 2026

Your Privacy Depends on Others: Collusion Vulnerabilities in Individual Differential Privacy

arXiv 2601.12922 ↗First posted 19 Jan 2026cs.CR cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Individual Differential Privacy (iDP) promises users control over their privacy, but this promise can be broken in practice. We reveal a previously overlooked vulnerability in sampling-based iDP mechanisms: while conforming to the iDP guarantees, an individual's privacy risk is not solely governed by their own privacy budget, but critically depends on the privacy choices of all other data contributors. This creates a mismatch between the promise of individual privacy control and the reality of a system where risk is collectively determined. We demonstrate empirically that certain distributions of privacy preferences can unintentionally inflate the privacy risk of individuals, even when their formal guarantees are met. Moreover, this excess risk provides an exploitable attack vector. A central adversary or a set of colluding adversaries can deliberately choose privacy budgets to amplify vulnerabilities of targeted individuals. Most importantly, this attack operates entirely within the guarantees of DP, hiding this excess vulnerability. Our empirical evaluation demonstrates successful attacks against 62% of targeted individuals, substantially increasing their membership inference susceptibility. To mitigate this, we propose (varepsiloni,δi,overlineDelta)-iDP a privacy contract that uses Delta-divergences to provide users with a hard upper bound on their excess vulnerability, while offering flexibility to mechanism design. Our findings expose a fundamental challenge to the current paradigm, demanding a re-evaluation of how iDP systems are designed, audited, communicated, and deployed to make excess risks transparent and controllable.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 35 of its 65 references that are in the map. First posted on arXiv 19 Jan 2026.

Builds on

35 of 65 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Deep Learning with Differential Privacycited by 2,682
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  4. 2021Membership Inference Attacks From First Principlescited by 839
  5. 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 682
  6. 2017Renyi Differential Privacycited by 643
  7. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  8. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  9. 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 273
  10. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 265
  11. 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 237
  12. 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 234
23 more
  1. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 193
  2. 2021On the Importance of Difficulty Calibration in Membership Inference Attackscited by 137
  3. 2022The Privacy Onion Effect: Memorization is Relativecited by 128
  4. 2023Privacy Auditing with One (1) Training Runcited by 119
  5. 2021Practical Blind Membership Inference Attack via Differential Comparisonscited by 73
  6. 2019Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?cited by 68
  7. 2020Individual Privacy Accounting via a Renyi Filtercited by 57
  8. 2024Membership Inference Attacks Cannot Prove that a Model Was Trained On Your Datacited by 49
  9. 2023Bounding Training Data Reconstruction in DP-SGDcited by 42
  10. 2016Privacy Odometers and Filters: Pay-as-you-Go Compositioncited by 32
  11. 2024Auditing f-Differential Privacy in One Runcited by 28
  12. 2022Fully Adaptive Composition in Differential Privacycited by 22
  13. 2025Scaling Laws for Differentially Private Language Modelscited by 22
  14. 2023Have it your way: Individualized Privacy Assignment for DP-SGDcited by 21
  15. 2022Individual Privacy Accounting for Differentially Private Stochastic Gradient Descentcited by 20
  16. 2022Individual Privacy Accounting with Gaussian Differential Privacycited by 19
  17. 2021"I need a better description'': An Investigation Into User Expectations For Differential Privacycited by 17
  18. 2025VaultGemma: A Differentially Private Gemma Modelcited by 14
  19. 2021Practical Privacy Filters and Odometers with R'enyi Differential Privacy and Applications to Differentially Private Deep Learningcited by 12
  20. 2025Unifying Re-Identification, Attribute Inference, and Data Reconstruction Risks in Differential Privacycited by 11
  21. 2024Beyond the Calibration Point: Mechanism Comparison in Differential Privacycited by 8
  22. 2025Optimizing Canaries for Privacy Auditing with Metagradient Descentcited by 6
  23. 2023Personalized DP-SGD using Sampling Mechanismscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.