# Bayesian Framework for Gradient Leakage

arXiv: https://arxiv.org/abs/2111.04706 · HTML: https://mlsanity.com/paper/2111.04706/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 8 Nov 2021.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Inverting Gradients -- How easy is it to break privacy in federated learning?](https://mlsanity.com/paper/2003.14053/) (2020)
- [Federated Optimization: Distributed Machine Learning for On-Device Intelligence](https://mlsanity.com/paper/1610.02527/) (2016)
- [Exploiting Unintended Feature Leakage in Collaborative Learning](https://mlsanity.com/paper/1805.04049/) (2018)
- [iDLG: Improved Deep Leakage from Gradients](https://mlsanity.com/paper/2001.02610/) (2020)
- [See through Gradients: Image Batch Recovery via GradInversion](https://mlsanity.com/paper/2104.07586/) (2021)
- [Gradient Inversion with Generative Image Prior](https://mlsanity.com/paper/2110.14962/) (2021)
- [Provable Defense against Privacy Leakage in Federated Learning from Representation Perspective](https://mlsanity.com/paper/2012.06043/) (2020)
- [Gradient-Leakage Resilient Federated Learning](https://mlsanity.com/paper/2107.01154/) (2021)
- [Towards General Deep Leakage in Federated Learning](https://mlsanity.com/paper/2110.09074/) (2021)
- [Privacy-preserving Collaborative Learning with Automatic Transformation Search](https://mlsanity.com/paper/2011.12505/) (2020)
- [PRECODE - A Generic Model Extension to Prevent Deep Gradient Leakage](https://mlsanity.com/paper/2108.04725/) (2021)

## Built on by (most cited first, 30 of 34)

- [LAMP: Extracting Text from Gradients with Language Model Priors](https://mlsanity.com/paper/2202.08827/) (2022)
- [A Survey on Gradient Inversion: Attacks, Defenses and Future Directions](https://mlsanity.com/paper/2206.07284/) (2022)
- [Gradient Obfuscation Gives a False Sense of Security in Federated Learning](https://mlsanity.com/paper/2206.04055/) (2022)
- [Reconstructing Training Data from Model Gradient, Provably](https://mlsanity.com/paper/2212.03714/) (2022)
- [Data Leakage in Federated Averaging](https://mlsanity.com/paper/2206.12395/) (2022)
- [Privacy and Fairness in Federated Learning: on the Perspective of Trade-off](https://mlsanity.com/paper/2306.14123/) (2023)
- [Accelerated Federated Learning with Decoupled Adaptive Optimization](https://mlsanity.com/paper/2207.07223/) (2022)
- [On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlook](https://mlsanity.com/paper/2307.16680/) (2023)
- [SPEAR:Exact Gradient Inversion of Batches in Federated Learning](https://mlsanity.com/paper/2403.03945/) (2024)
- [TabLeak: Tabular Data Leakage in Federated Learning](https://mlsanity.com/paper/2210.01785/) (2022)
- [Inf2Guard: An Information-Theoretic Framework for Learning Privacy-Preserving Representations against Inference Attacks](https://mlsanity.com/paper/2403.02116/) (2024)
- [Surrogate Model Extension (SME): A Fast and Accurate Weight Update Attack on Federated Learning](https://mlsanity.com/paper/2306.00127/) (2023)
- [Hiding in Plain Sight: Disguising Data Stealing Attacks in Federated Learning](https://mlsanity.com/paper/2306.03013/) (2023)
- [Expressive variational quantum circuits provide inherent privacy in federated learning](https://mlsanity.com/paper/2309.13002/) (2023)
- [Understanding Deep Gradient Leakage via Inversion Influence Functions](https://mlsanity.com/paper/2309.13016/) (2023)
- [Data Reconstruction Attacks and Defenses: A Systematic Evaluation](https://mlsanity.com/paper/2402.09478/) (2024)
- [Towards Eliminating Hard Label Constraints in Gradient Inversion Attacks](https://mlsanity.com/paper/2402.03124/) (2024)
- [From Principle to Practice: Vertical Data Minimization for Machine Learning](https://mlsanity.com/paper/2311.10500/) (2023)
- [GI-PIP: Do We Require Impractical Auxiliary Dataset for Gradient Inversion Attacks?](https://mlsanity.com/paper/2401.11748/) (2024)
- [Seeing the Forest through the Trees: Data Leakage from Partial Transformer Gradients](https://mlsanity.com/paper/2406.00999/) (2024)
- [A Sample-Level Evaluation and Generative Framework for Model Inversion Attacks](https://mlsanity.com/paper/2502.19070/) (2025)
- [Defense Against Gradient Leakage Attacks via Learning to Obscure Data](https://mlsanity.com/paper/2206.00769/) (2022)
- [Combining Stochastic Defenses to Resist Gradient Inversion: An Ablation Study](https://mlsanity.com/paper/2208.04767/) (2022)
- [Refiner: Data Refining against Gradient Leakage Attacks in Federated Learning](https://mlsanity.com/paper/2212.02042/) (2022)
- [Privacy Preserving Federated Learning with Convolutional Variational Bottlenecks](https://mlsanity.com/paper/2309.04515/) (2023)
- [Exploring User-level Gradient Inversion with a Diffusion Prior](https://mlsanity.com/paper/2409.07291/) (2024)
- [SVDefense: Effective Defense against Gradient Inversion Attacks via Singular Value Decomposition](https://mlsanity.com/paper/2510.03319/) (2025)
- [BACON: Bayesian Optimal Condensation Framework for Dataset Distillation](https://mlsanity.com/paper/2406.01112/) (2024)
- [Improving Generalization in Heterogeneous Federated Continual Learning via Spatio-Temporal Gradient Matching with Prototypical Coreset](https://mlsanity.com/paper/2506.12031/) (2025)
- [Understanding Data Reconstruction Leakage in Federated Learning from a Theoretical Perspective](https://mlsanity.com/paper/2408.12119/) (2024)

## Abstract (arXiv)

Federated learning is an established method for training machine learning models without sharing training data. However, recent work has shown that it cannot guarantee data privacy as shared gradients can still leak sensitive information. To formalize the problem of gradient leakage, we propose a theoretical framework that enables, for the first time, analysis of the Bayes optimal adversary phrased as an optimization problem. We demonstrate that existing leakage attacks can be seen as approximations of this optimal adversary with different assumptions on the probability distributions of the input data and gradients. Our experiments confirm the effectiveness of the Bayes optimal adversary when it has knowledge of the underlying distribution. Further, our experimental evaluation shows that several existing heuristic defenses are not effective against stronger attacks, especially early in the training process. Thus, our findings indicate that the construction of more effective defenses and their evaluation remains an open problem.