# Subject Membership Inference Attacks in Federated Learning

arXiv: https://arxiv.org/abs/2206.03317 · HTML: https://mlsanity.com/paper/2206.03317/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security.
- Builds on 29 of its 58 references that are in the map.
- First posted on arXiv 7 Jun 2022.

## Placed in

- LLM safety and security / [Federated unlearning and poisoning](https://mlsanity.com/safety/federated-unlearning-and-poisoning/)

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [Inverting Gradients -- How easy is it to break privacy in federated learning?](https://mlsanity.com/paper/2003.14053/) (2020)
- [Train faster, generalize better: Stability of stochastic gradient descent](https://mlsanity.com/paper/1509.01240/) (2015)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning](https://mlsanity.com/paper/1702.07464/) (2017)
- [Enhanced Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/2111.09679/) (2021)
- [Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning](https://mlsanity.com/paper/1812.00535/) (2018)
- [The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks](https://mlsanity.com/paper/1911.07135/) (2019)
- [Evaluating Gradient Inversion Attacks and Defenses in Federated Learning](https://mlsanity.com/paper/2112.00059/) (2021)
- [LDP-Fed: Federated Learning with Local Differential Privacy](https://mlsanity.com/paper/2006.03637/) (2020)
- [Hacking Smart Machines with Smarter Ones: How to Extract Meaningful Data from Machine Learning Classifiers](https://mlsanity.com/paper/1306.4447/) (2013)
- [Revisiting Membership Inference Under Realistic Assumptions](https://mlsanity.com/paper/2005.10881/) (2020)
- [ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models](https://mlsanity.com/paper/2102.02551/) (2021)
- [Survey: Leakage and Privacy at Inference Time](https://mlsanity.com/paper/2107.01614/) (2021)
- [Geographic Spines in the 2020 Census Disclosure Avoidance System](https://mlsanity.com/paper/2203.16654/) (2022)
- [Formalizing and Estimating Distribution Inference Risks](https://mlsanity.com/paper/2109.06024/) (2021)
- [Learning discrete distributions: user vs item-level privacy](https://mlsanity.com/paper/2007.13660/) (2020)
- [Membership Inference on Word Embedding and Beyond](https://mlsanity.com/paper/2106.11384/) (2021)
- [Layer-wise Characterization of Latent Information Leakage in Federated Learning](https://mlsanity.com/paper/2010.08762/) (2020)
- [A Quantitative Metric for Privacy Leakage in Federated Learning](https://mlsanity.com/paper/2102.13472/) (2021)

## Built on by (most cited first, 20 of 20)

- [SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning](https://mlsanity.com/paper/2212.10986/) (2022)
- [Distribution inference risks: Identifying and mitigating sources of leakage](https://mlsanity.com/paper/2209.08541/) (2022)
- [Privacy-Aware Document Visual Question Answering](https://mlsanity.com/paper/2312.10108/) (2023)
- [Dissecting Distribution Inference](https://mlsanity.com/paper/2212.07591/) (2022)
- [FedMIA: An Effective Membership Inference Attack Exploiting "All for One" Principle in Federated Learning](https://mlsanity.com/paper/2402.06289/) (2024)
- [Manipulating Transfer Learning for Property Inference](https://mlsanity.com/paper/2303.11643/) (2023)
- [From Centralized to Decentralized Federated Learning: Theoretical Insights, Privacy Preservation, and Robustness Challenges](https://mlsanity.com/paper/2503.07505/) (2025)
- [FedCSD: A Federated Learning Based Approach for Code-Smell Detection](https://mlsanity.com/paper/2306.00038/) (2023)
- [Subject Data Auditing via Source Inference Attack in Cross-Silo Federated Learning](https://mlsanity.com/paper/2409.19417/) (2024)
- [Research in Collaborative Learning Does Not Serve Cross-Silo Federated Learning in Practice](https://mlsanity.com/paper/2510.12595/) (2025)
- [Security and Privacy Issues of Federated Learning](https://mlsanity.com/paper/2307.12181/) (2023)
- [Privacy-Preserving Data Deduplication for Enhancing Federated Learning of Language Models (Extended Version)](https://mlsanity.com/paper/2407.08152/) (2024)
- [Efficient Federated Unlearning under Plausible Deniability](https://mlsanity.com/paper/2410.09947/) (2024)
- [NeurIPS 2023 Competition: Privacy Preserving Federated Learning Document VQA](https://mlsanity.com/paper/2411.03730/) (2024)
- [Securing Genomic Data Against Inference Attacks in Federated Learning Environments](https://mlsanity.com/paper/2505.07188/) (2025)
- [United We Defend: Collaborative Membership Inference Defenses in Federated Learning](https://mlsanity.com/paper/2601.06866/) (2026)
- [Black-Box Privacy Attacks on Shared Representations in Multitask Learning](https://mlsanity.com/paper/2506.16460/) (2025)
- [Enhancing Federated Learning Privacy with QUBO](https://mlsanity.com/paper/2511.02785/) (2025)
- [Res-MIA: A Training-Free Resolution-Based Membership Inference Attack on Federated Learning Models](https://mlsanity.com/paper/2601.17378/) (2026)
- [Kinematics-Induced Multimodal 3D Human Pose Estimation with Subject-Level Privacy](https://mlsanity.com/paper/2610.02943/) (2026)

## Abstract (arXiv)

Privacy attacks on Machine Learning (ML) models often focus on inferring the existence of particular data points in the training data. However, what the adversary really wants to know is if a particular individual's (subject's) data was included during training. In such scenarios, the adversary is more likely to have access to the distribution of a particular subject than actual records. Furthermore, in settings like cross-silo Federated Learning (FL), a subject's data can be embodied by multiple data records that are spread across multiple organizations. Nearly all of the existing private FL literature is dedicated to studying privacy at two granularities -- item-level (individual data records), and user-level (participating user in the federation), neither of which apply to data subjects in cross-silo FL. This insight motivates us to shift our attention from the privacy of data records to the privacy of data subjects, also known as subject-level privacy. We propose two novel black-box attacks for subject membership inference, of which one assumes access to a model after each training round. Using these attacks, we estimate subject membership inference risk on real-world data for single-party models as well as FL scenarios. We find our attacks to be extremely potent, even without access to exact training records, and using the knowledge of membership for a handful of subjects. To better understand the various factors that may influence subject privacy risk in cross-silo FL settings, we systematically generate several hundred synthetic federation configurations, varying properties of the data, model design and training, and the federation itself. Finally, we investigate the effectiveness of Differential Privacy in mitigating this threat.