# Dropout against Deep Leakage from Gradients

arXiv: https://arxiv.org/abs/2108.11106 · HTML: https://mlsanity.com/paper/2108.11106/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 11 references that are in the map.
- First posted on arXiv 25 Aug 2021.

## Builds on (most cited first, 5 of 5)

- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [iDLG: Improved Deep Leakage from Gradients](https://mlsanity.com/paper/2001.02610/) (2020)
- [TextHide: Tackling Data Privacy in Language Understanding Tasks](https://mlsanity.com/paper/2010.06053/) (2020)

## Built on by (most cited first, 4 of 4)

- [A Survey on Gradient Inversion: Attacks, Defenses and Future Directions](https://mlsanity.com/paper/2206.07284/) (2022)
- [Dropout is NOT All You Need to Prevent Gradient Leakage](https://mlsanity.com/paper/2208.06163/) (2022)
- [FEDLAD: Federated Evaluation of Deep Leakage Attacks and Defenses](https://mlsanity.com/paper/2411.03019/) (2024)
- [TS-Inverse: A Gradient Inversion Attack Tailored for Federated Time Series Forecasting Models](https://mlsanity.com/paper/2503.20952/) (2025)

## Abstract (arXiv)

As the scale and size of the data increases significantly nowadays, federal learning (Bonawitz et al. [2019]) for high performance computing and machine learning has been much more important than ever before (Abadi et al. [2016]). People used to believe that sharing gradients seems to be safe to conceal the local training data during the training stage. However, Zhu et al. [2019] demonstrated that it was possible to recover raw data from the model training data by detecting gradients. They use generated random dummy data and minimise the distance between them and real data. Zhao et al. [2020] pushes the convergence algorithm even further. By replacing the original loss function with cross entropy loss, they achieve better fidelity threshold. In this paper, we propose using an additional dropout (Srivastava et al. [2014]) layer before feeding the data to the classifier. It is very effective in preventing leakage of raw data, as the training data cannot converge to a small RMSE even after 5,800 epochs with dropout rate set to 0.5.