# The Federation Strikes Back: A Survey of Federated Learning Privacy Attacks, Defenses, Applications, and Policy Landscape

arXiv: https://arxiv.org/abs/2405.03636 · HTML: https://mlsanity.com/paper/2405.03636/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 81 of its 146 references that are in the map.
- First posted on arXiv 6 May 2024.

## Builds on (most cited first, 30 of 81)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Inverting Gradients -- How easy is it to break privacy in federated learning?](https://mlsanity.com/paper/2003.14053/) (2020)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [The Future of Digital Health with Federated Learning](https://mlsanity.com/paper/2003.08119/) (2020)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [Federated Learning with Differential Privacy: Algorithms and Performance Analysis](https://mlsanity.com/paper/1911.00222/) (2019)
- [Exploiting Unintended Feature Leakage in Collaborative Learning](https://mlsanity.com/paper/1805.04049/) (2018)
- [Adaptive Federated Learning in Resource Constrained Edge Computing Systems](https://mlsanity.com/paper/1804.05271/) (2018)
- [iDLG: Improved Deep Leakage from Gradients](https://mlsanity.com/paper/2001.02610/) (2020)
- [A Unified Theory of Decentralized SGD with Changing Topology and Local Updates](https://mlsanity.com/paper/2003.10422/) (2020)
- [HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients](https://mlsanity.com/paper/2010.01264/) (2020)
- [Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning](https://mlsanity.com/paper/1702.07464/) (2017)
- [See through Gradients: Image Batch Recovery via GradInversion](https://mlsanity.com/paper/2104.07586/) (2021)
- [Knockoff Nets: Stealing Functionality of Black-Box Models](https://mlsanity.com/paper/1812.02766/) (2018)
- [Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning](https://mlsanity.com/paper/1812.00535/) (2018)
- [Enhancing the Transferability of Adversarial Attacks through Variance Tuning](https://mlsanity.com/paper/2103.15571/) (2021)
- [Unlocking High-Accuracy Differentially Private Image Classification through Scale](https://mlsanity.com/paper/2204.13650/) (2022)
- [Federated learning with hierarchical clustering of local updates to improve training on non-IID data](https://mlsanity.com/paper/2004.11791/) (2020)
- [Client-Edge-Cloud Hierarchical Federated Learning](https://mlsanity.com/paper/1905.06641/) (2019)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [A Survey of Privacy Attacks in Machine Learning](https://mlsanity.com/paper/2007.07646/) (2020)
- [When the Curious Abandon Honesty: Federated Learning Is Not Private](https://mlsanity.com/paper/2112.02918/) (2021)
- [Privacy Auditing with One (1) Training Run](https://mlsanity.com/paper/2305.08846/) (2023)
- [Stealing Hyperparameters in Machine Learning](https://mlsanity.com/paper/1802.05351/) (2018)

## Built on by (most cited first, 13 of 13)

- [Energy-Latency Attacks: A New Adversarial Threat to Deep Learning](https://mlsanity.com/paper/2503.04963/) (2025)
- [When Federated Learning Meets Quantum Computing: Survey and Research Opportunities](https://mlsanity.com/paper/2504.08814/) (2025)
- [The MICCAI Federated Tumor Segmentation (FeTS) Challenge 2024: Efficient and Robust Aggregation Methods for Federated Learning](https://mlsanity.com/paper/2512.06206/) (2025)
- [Beyond Corner Patches: Semantics-Aware Backdoor Attack in Federated Learning](https://mlsanity.com/paper/2603.29328/) (2026)
- [Securing Genomic Data Against Inference Attacks in Federated Learning Environments](https://mlsanity.com/paper/2505.07188/) (2025)
- [FedVSR: Towards Model-Agnostic Federated Learning in Video Super-Resolution](https://mlsanity.com/paper/2503.13745/) (2025)
- [FedThief: Harming Others to Benefit Oneself in Self-Centered Federated Learning](https://mlsanity.com/paper/2509.00540/) (2025)
- [FedPoisonTTP: A Threat Model and Poisoning Attack for Federated Test-Time Personalization](https://mlsanity.com/paper/2511.19248/) (2025)
- [Trust-Based Incentive Mechanisms in Semi-Decentralized Federated Learning Systems](https://mlsanity.com/paper/2602.08290/) (2026)
- [Rashomon Sets and Model Multiplicity in Federated Learning](https://mlsanity.com/paper/2602.09520/) (2026)
- [AI Security Research Should Better Incentivize Defense Research](https://mlsanity.com/paper/2605.23448/) (2026)
- [Color Matters: Trigger Color Affects Success in Federated Backdoor Attacks](https://mlsanity.com/paper/2606.25858/) (2026)
- [Privacy-Preserving Action Recognition: Taxonomy, Methods, and Privacy-Utility Trade-offs](https://mlsanity.com/paper/2608.04501/) (2026)

## Abstract (arXiv)

Deep learning has shown incredible potential across a wide array of tasks, and accompanied by this growth has been an insatiable appetite for data. However, a large amount of data needed for enabling deep learning is stored on personal devices, and recent concerns on privacy have further highlighted challenges for accessing such data. As a result, federated learning (FL) has emerged as an important privacy-preserving technology that enables collaborative training of machine learning models without the need to send the raw, potentially sensitive, data to a central server. However, the fundamental premise that sending model updates to a server is privacy-preserving only holds if the updates cannot be "reverse engineered" to infer information about the private training data. It has been shown under a wide variety of settings that this privacy premise does not hold. In this survey paper, we provide a comprehensive literature review of the different privacy attacks and defense methods in FL. We identify the current limitations of these attacks and highlight the settings in which the privacy of an FL client can be broken. We further dissect some of the successful industry applications of FL and draw lessons for future successful adoption. We survey the emerging landscape of privacy regulation for FL and conclude with future directions for taking FL toward the cherished goal of generating accurate models while preserving the privacy of the data from its participants.