mlsanityData to 14 Oct 2026

Subject Membership Inference Attacks in Federated Learning

arXiv 2206.03317 ↗First posted 7 Jun 2022cs.LG cs.AI cs.CRcited by 20

On the map: LLM safety and security → Federated unlearning and poisoning

Abstract (arXiv)

Privacy attacks on Machine Learning (ML) models often focus on inferring the existence of particular data points in the training data. However, what the adversary really wants to know is if a particular individual's (subject's) data was included during training. In such scenarios, the adversary is more likely to have access to the distribution of a particular subject than actual records. Furthermore, in settings like cross-silo Federated Learning (FL), a subject's data can be embodied by multiple data records that are spread across multiple organizations. Nearly all of the existing private FL literature is dedicated to studying privacy at two granularities -- item-level (individual data records), and user-level (participating user in the federation), neither of which apply to data subjects in cross-silo FL. This insight motivates us to shift our attention from the privacy of data records to the privacy of data subjects, also known as subject-level privacy. We propose two novel black-box attacks for subject membership inference, of which one assumes access to a model after each training round. Using these attacks, we estimate subject membership inference risk on real-world data for single-party models as well as FL scenarios. We find our attacks to be extremely potent, even without access to exact training records, and using the knowledge of membership for a handful of subjects. To better understand the various factors that may influence subject privacy risk in cross-silo FL settings, we systematically generate several hundred synthetic federation configurations, varying properties of the data, model design and training, and the federation itself. Finally, we investigate the effectiveness of Differential Privacy in mitigating this threat.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security. Builds on 29 of its 58 references that are in the map. First posted on arXiv 7 Jun 2022.

Builds on

29 of 58 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2016Deep Learning with Differential Privacycited by 2,728
  5. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  6. 2016An overview of gradient descent optimization algorithmscited by 1,033
  7. 2019Deep Leakage from Gradientscited by 1,021
  8. 2018LEAF: A Benchmark for Federated Settingscited by 792
  9. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
  10. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  11. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
  12. 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 552
17 more
  1. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 498
  2. 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 334
  3. 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 248
  4. 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 246
  5. 2019The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networkscited by 246
  6. 2021Evaluating Gradient Inversion Attacks and Defenses in Federated Learningcited by 196
  7. 2020LDP-Fed: Federated Learning with Local Differential Privacycited by 132
  8. 2013Hacking Smart Machines with Smarter Ones: How to Extract Meaningful Data from Machine Learning Classifierscited by 130
  9. 2020Revisiting Membership Inference Under Realistic Assumptionscited by 112
  10. 2021ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Modelscited by 85
  11. 2021Survey: Leakage and Privacy at Inference Timecited by 44
  12. 2022Geographic Spines in the 2020 Census Disclosure Avoidance Systemcited by 44
  13. 2021Formalizing and Estimating Distribution Inference Riskscited by 43
  14. 2020Learning discrete distributions: user vs item-level privacycited by 41
  15. 2021Membership Inference on Word Embedding and Beyondcited by 35
  16. 2020Layer-wise Characterization of Latent Information Leakage in Federated Learningcited by 19
  17. 2021A Quantitative Metric for Privacy Leakage in Federated Learningcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 4; 2024: 5; 2025: 5; 2026: 3.

20 papers.

Sort:

By area:

  1. 2022SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learningcited by 51
  2. 2022Distribution inference risks: Identifying and mitigating sources of leakagecited by 17
  3. 2023Privacy-Aware Document Visual Question Answeringcited by 17
  4. 2022Dissecting Distribution Inferencecited by 15
  5. 2024FedMIA: An Effective Membership Inference Attack Exploiting "All for One" Principle in Federated Learningcited by 12
  6. 2023Manipulating Transfer Learning for Property Inferencecited by 8
  7. 2025From Centralized to Decentralized Federated Learning: Theoretical Insights, Privacy Preservation, and Robustness Challengescited by 4
  8. 2023FedCSD: A Federated Learning Based Approach for Code-Smell Detectioncited by 3
  9. 2024Subject Data Auditing via Source Inference Attack in Cross-Silo Federated Learningcited by 3
  10. 2025Research in Collaborative Learning Does Not Serve Cross-Silo Federated Learning in Practicecited by 3
  11. 2023Security and Privacy Issues of Federated Learningcited by 2
  12. 2024Privacy-Preserving Data Deduplication for Enhancing Federated Learning of Language Models (Extended Version)cited by 2
8 more
  1. 2024Efficient Federated Unlearning under Plausible Deniabilitycited by 2
  2. 2024NeurIPS 2023 Competition: Privacy Preserving Federated Learning Document VQAcited by 2
  3. 2025Securing Genomic Data Against Inference Attacks in Federated Learning Environmentscited by 1
  4. 2026United We Defend: Collaborative Membership Inference Defenses in Federated Learningcited by 1
  5. 2025Black-Box Privacy Attacks on Shared Representations in Multitask Learningcited by 0
  6. 2025Enhancing Federated Learning Privacy with QUBOcited by 0
  7. 2026Res-MIA: A Training-Free Resolution-Based Membership Inference Attack on Federated Learning Modelscited by 0
  8. 2026Kinematics-Induced Multimodal 3D Human Pose Estimation with Subject-Level Privacycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Membership Inference Attacks and Defenses in Federated Learning: A Survey
  2. 2023Source Inference Attacks: Beyond Membership Inference Attacks in Federated Learning
  3. 2021Source Inference Attacks in Federated Learning
  4. 2024Privacy in Federated Learning
  5. 2023Active Membership Inference Attack under Local Differential Privacy in Federated Learning
  6. 2023MIA-BAD: An Approach for Enhancing Membership Inference Attack and its Mitigation with Federated Learning
  7. 2024Accuracy-Privacy Trade-off in the Mitigation of Membership Inference Attack in Federated Learning
  8. 2026SoK: Challenges in Tabular Membership Inference Attacks