# Understanding Training-Data Leakage from Gradients in Neural Networks for Image Classification

arXiv: https://arxiv.org/abs/2111.10178 · HTML: https://mlsanity.com/paper/2111.10178/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 11 references that are in the map.
- First posted on arXiv 19 Nov 2021.

## Builds on (most cited first, 6 of 6)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [iDLG: Improved Deep Leakage from Gradients](https://mlsanity.com/paper/2001.02610/) (2020)
- [R-GAP: Recursive Gradient Attack on Privacy](https://mlsanity.com/paper/2010.07733/) (2020)
- [Rethinking Privacy Preserving Deep Learning: How to Evaluate and Thwart Privacy Attacks](https://mlsanity.com/paper/2006.11601/) (2020)

## Built on by (most cited first, 16 of 16)

- [GIFD: A Generative Gradient Inversion Method with Feature Domain Optimization](https://mlsanity.com/paper/2308.04699/) (2023)
- [A Survey on Gradient Inversion: Attacks, Defenses and Future Directions](https://mlsanity.com/paper/2206.07284/) (2022)
- [Reconstructing Training Data from Model Gradient, Provably](https://mlsanity.com/paper/2212.03714/) (2022)
- [Privacy Meets Explainability: A Comprehensive Impact Benchmark](https://mlsanity.com/paper/2211.04110/) (2022)
- [Understanding Deep Gradient Leakage via Inversion Influence Functions](https://mlsanity.com/paper/2309.13016/) (2023)
- [Representation Magnitude has a Liability to Privacy Vulnerability](https://mlsanity.com/paper/2407.16164/) (2024)
- [Contextual Combinatorial Multi-output GP Bandits with Group Constraints](https://mlsanity.com/paper/2111.14778/) (2021)
- [AFGI: Towards Accurate and Fast-convergent Gradient Inversion Attack in Federated Learning](https://mlsanity.com/paper/2403.08383/) (2024)
- [Dealing Doubt: Unveiling Threat Models in Gradient Inversion Attacks under Federated Learning, A Survey and Taxonomy](https://mlsanity.com/paper/2405.10376/) (2024)
- [Model Inversion Attack against Federated Unlearning](https://mlsanity.com/paper/2502.14558/) (2025)
- [Enhancing Gradient Inversion Attacks in Federated Learning via Hierarchical Feature Optimization](https://mlsanity.com/paper/2604.00955/) (2026)
- [MGIC: A Multi-Label Gradient Inversion Attack based on Canny Edge Detection on Federated Learning](https://mlsanity.com/paper/2403.08284/) (2024)
- [R-CONV: An Analytical Approach for Efficient Data Reconstruction via Convolutional Gradients](https://mlsanity.com/paper/2406.04227/) (2024)
- [Optimal Defenses Against Gradient Reconstruction Attacks](https://mlsanity.com/paper/2411.03746/) (2024)
- [FedEM: A Privacy-Preserving Framework for Concurrent Utility Preservation in Federated Learning](https://mlsanity.com/paper/2503.06021/) (2025)
- [Images in Motion?: A First Look into Video Leakage in Collaborative Deep Learning](https://mlsanity.com/paper/2509.09742/) (2025)

## Abstract (arXiv)

Federated learning of deep learning models for supervised tasks, e.g. image classification and segmentation, has found many applications: for example in human-in-the-loop tasks such as film post-production where it enables sharing of domain expertise of human artists in an efficient and effective fashion. In many such applications, we need to protect the training data from being leaked when gradients are shared in the training process due to IP or privacy concerns. Recent works have demonstrated that it is possible to reconstruct the training data from gradients for an image-classification model when its architecture is known. However, there is still an incomplete theoretical understanding of the efficacy and failure of such attacks. In this paper, we analyse the source of training-data leakage from gradients. We formulate the problem of training data reconstruction as solving an optimisation problem iteratively for each layer. The layer-wise objective function is primarily defined by weights and gradients from the current layer as well as the output from the reconstruction of the subsequent layer, but it might also involve a 'pull-back' constraint from the preceding layer. Training data can be reconstructed when we solve the problem backward from the output of the network through each layer. Based on this formulation, we are able to attribute the potential leakage of the training data in a deep network to its architecture. We also propose a metric to measure the level of security of a deep learning model against gradient-based attacks on the training data.