# PRECODE - A Generic Model Extension to Prevent Deep Gradient Leakage

arXiv: https://arxiv.org/abs/2108.04725 · HTML: https://mlsanity.com/paper/2108.04725/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 37 references that are in the map.
- First posted on arXiv 10 Aug 2021.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Inverting Gradients -- How easy is it to break privacy in federated learning?](https://mlsanity.com/paper/2003.14053/) (2020)
- [Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training](https://mlsanity.com/paper/1712.01887/) (2017)
- [Robust and Communication-Efficient Federated Learning from Non-IID Data](https://mlsanity.com/paper/1903.02891/) (2019)
- [iDLG: Improved Deep Leakage from Gradients](https://mlsanity.com/paper/2001.02610/) (2020)
- [Privacy-preserving Federated Brain Tumour Segmentation](https://mlsanity.com/paper/1910.00962/) (2019)
- [BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learning](https://mlsanity.com/paper/1905.06731/) (2019)
- [R-GAP: Recursive Gradient Attack on Privacy](https://mlsanity.com/paper/2010.07733/) (2020)
- [A Framework for Evaluating Gradient Leakage Attacks in Federated Learning](https://mlsanity.com/paper/2004.10397/) (2020)
- [SAPAG: A Self-Adaptive Privacy Attack From Gradients](https://mlsanity.com/paper/2009.06228/) (2020)
- [TornadoAggregate: Accurate and Scalable Federated Learning via the Ring-Based Architecture](https://mlsanity.com/paper/2012.03214/) (2020)
- [A Transprecision Floating-Point Platform for Ultra-Low Power Computing](https://mlsanity.com/paper/1711.10374/) (2017)

## Built on by (most cited first, 28 of 28)

- [LAMP: Extracting Text from Gradients with Language Model Priors](https://mlsanity.com/paper/2202.08827/) (2022)
- [Gradient Obfuscation Gives a False Sense of Security in Federated Learning](https://mlsanity.com/paper/2206.04055/) (2022)
- [Bayesian Framework for Gradient Leakage](https://mlsanity.com/paper/2111.04706/) (2021)
- [Data Leakage in Federated Averaging](https://mlsanity.com/paper/2206.12395/) (2022)
- [Privacy and Fairness in Federated Learning: on the Perspective of Trade-off](https://mlsanity.com/paper/2306.14123/) (2023)
- [Inf2Guard: An Information-Theoretic Framework for Learning Privacy-Preserving Representations against Inference Attacks](https://mlsanity.com/paper/2403.02116/) (2024)
- [Dropout is NOT All You Need to Prevent Gradient Leakage](https://mlsanity.com/paper/2208.06163/) (2022)
- [A Framework for Evaluating Privacy-Utility Trade-off in Vertical Federated Learning](https://mlsanity.com/paper/2209.03885/) (2022)
- [A Survey on Vulnerability of Federated Learning: A Learning Algorithm Perspective](https://mlsanity.com/paper/2311.16065/) (2023)
- [Exploring the Vulnerabilities of Federated Learning: A Deep Dive into Gradient Inversion Attacks](https://mlsanity.com/paper/2503.11514/) (2025)
- [Understanding Deep Gradient Leakage via Inversion Influence Functions](https://mlsanity.com/paper/2309.13016/) (2023)
- [Gradient Leakage Defense with Key-Lock Module for Federated Learning](https://mlsanity.com/paper/2305.04095/) (2023)
- [Toward the Tradeoffs between Privacy, Fairness and Utility in Federated Learning](https://mlsanity.com/paper/2311.18190/) (2023)
- [Analyzing Inference Privacy Risks Through Gradients in Machine Learning](https://mlsanity.com/paper/2408.16913/) (2024)
- [Combining Stochastic Defenses to Resist Gradient Inversion: An Ablation Study](https://mlsanity.com/paper/2208.04767/) (2022)
- [Privacy Preserving Federated Learning with Convolutional Variational Bottlenecks](https://mlsanity.com/paper/2309.04515/) (2023)
- [Gradients Stand-in for Defending Deep Leakage in Federated Learning](https://mlsanity.com/paper/2410.08734/) (2024)
- [SVDefense: Effective Defense against Gradient Inversion Attacks via Singular Value Decomposition](https://mlsanity.com/paper/2510.03319/) (2025)
- [Directional Privacy for Deep Learning](https://mlsanity.com/paper/2211.04686/) (2022)
- [FEDLAD: Federated Evaluation of Deep Leakage Attacks and Defenses](https://mlsanity.com/paper/2411.03019/) (2024)
- [Empirical Calibration and Metric Differential Privacy in Language Models](https://mlsanity.com/paper/2503.13872/) (2025)
- [Deep Leakage with Generative Flow Matching Denoiser](https://mlsanity.com/paper/2601.15049/) (2026)
- [DRAUN: An Algorithm-Agnostic Data Reconstruction Attack on Federated Unlearning Systems](https://mlsanity.com/paper/2506.01777/) (2025)
- [Evaluating Selective Encryption Against Gradient Inversion Attacks](https://mlsanity.com/paper/2508.04155/) (2025)
- [FedHypeVAE: Federated Learning with Hypernetwork Generated Conditional VAEs for Differentially Private Embedding Sharing](https://mlsanity.com/paper/2601.00785/) (2026)
- [SpooFL: Spoofing Federated Learning](https://mlsanity.com/paper/2601.15055/) (2026)
- [TriShield: Zero-Utility-Loss Defense Against Privacy Backdoors in Federated Language Model Fine-Tuning via Orthogonal Gradient Projection and Optimizer State Entanglement](https://mlsanity.com/paper/2607.27940/) (2026)
- [A Privacy Study of Sparse Collaborative Inference](https://mlsanity.com/paper/2608.16236/) (2026)

## Abstract (arXiv)

Collaborative training of neural networks leverages distributed data by exchanging gradient information between different clients. Although training data entirely resides with the clients, recent work shows that training data can be reconstructed from such exchanged gradient information. To enhance privacy, gradient perturbation techniques have been proposed. However, they come at the cost of reduced model performance, increased convergence time, or increased data demand. In this paper, we introduce PRECODE, a PRivacy EnhanCing mODulE that can be used as generic extension for arbitrary model architectures. We propose a simple yet effective realization of PRECODE using variational modeling. The stochastic sampling induced by variational modeling effectively prevents privacy leakage from gradients and in turn preserves privacy of data owners. We evaluate PRECODE using state of the art gradient inversion attacks on two different model architectures trained on three datasets. In contrast to commonly used defense mechanisms, we find that our proposed modification consistently reduces the attack success rate to 0% while having almost no negative impact on model training and final performance. As a result, PRECODE reveals a promising path towards privacy enhancing model extensions.