mlsanityData to 7 Oct 2026

See through Gradients: Image Batch Recovery via GradInversion

arXiv 2104.07586 ↗First posted 15 Apr 2021cs.LG cs.CVcited by 320

Not on the map. Why?

Abstract (arXiv)

Training deep neural networks requires gradient estimation from data batches to update parameters. Gradients per parameter are averaged over a set of data and this has been presumed to be safe for privacy-preserving training in joint, collaborative, and federated learning applications. Prior work only showed the possibility of recovering input data given gradients under very restrictive conditions - a single input point, or a network with no non-linearities, or a small 32x32 px input batch. Therefore, averaging gradients over larger batches was thought to be safe. In this work, we introduce GradInversion, using which input images from a larger batch (8 - 48 images) can also be recovered for large networks such as ResNets (50 layers), on complex datasets such as ImageNet (1000 classes, 224x224 px). We formulate an optimization task that converts random noise into natural images, matching gradients while regularizing image fidelity. We also propose an algorithm for target class label recovery given gradients. We further propose a group consistency regularization framework, where multiple agents starting from different random seeds work together to find an enhanced reconstruction of original data batch. We show that gradients encode a surprisingly large amount of information, such that all the individual images can be recovered with high fidelity via GradInversion, even for complex datasets, deep networks, and large batch sizes.

Key facts. Cited by 320 papers in the map (data through 7 Oct 2026). Builds on 41 of its 56 references that are in the map. First posted on arXiv 15 Apr 2021.

Builds on

41 of 56 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  7. 2016Pyramid Scene Parsing Networkcited by 2,472
  8. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  9. 2017Improved Training of Wasserstein GANscited by 2,223
  10. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  11. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  12. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
29 more
  1. 2017Mixed Precision Trainingcited by 1,135
  2. 2019Deep Leakage from Gradientscited by 1,004
  3. 2019Towards Federated Learning at Scale: System Designcited by 826
  4. 2018Self-Attention Generative Adversarial Networkscited by 809
  5. 2018Path Aggregation Network for Instance Segmentationcited by 796
  6. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  7. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  8. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
  9. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
  10. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  11. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 402
  12. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  13. 2020iDLG: Improved Deep Leakage from Gradientscited by 367
  14. 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 326
  15. 2018Context Encoding for Semantic Segmentationcited by 289
  16. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 265
  17. 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 242
  18. 2019The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networkscited by 241
  19. 2019Data-Free Learning of Student Networkscited by 203
  20. 2016Synthesizing the preferred inputs for neurons in neural networks via deep generator networkscited by 181
  21. 2019Zero-shot Knowledge Transfer via Adversarial Belief Matchingcited by 129
  22. 2015Visualizing Deep Convolutional Neural Networks Using Natural Pre-Imagescited by 93
  23. 2019The Knowledge Within: Methods for Data-Free Model Compressioncited by 54
  24. 2020RANSAC-Flow: generic two-stage image alignmentcited by 48
  25. 2019Inverting Deep Generative models, One layer at a timecited by 33
  26. 2017Learning Energy-Based Models as Generative ConvNets via Multi-grid Modeling and Samplingcited by 32
  27. 2018Federated Learning for Ultra-Reliable Low-Latency V2V Communicationscited by 30
  28. 2019Adversarial Neural Network Inversion via Auxiliary Knowledge Alignmentcited by 30
  29. 2015FireCaffe: near-linear acceleration of deep neural network training on compute clusterscited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 28; 2022: 70; 2023: 70; 2024: 71; 2025: 53; 2026: 28.

320 papers.

Sort:

By area:

  1. 2021Evaluating Gradient Inversion Attacks and Defenses in Federated Learningcited by 196
  2. 2022Reconstructing Training Data from Trained Neural Networkscited by 129
  3. 2021When the Curious Abandon Honesty: Federated Learning Is Not Privatecited by 126
  4. 2022Differentially Private Diffusion Modelscited by 118
  5. 2021Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Modelscited by 106
  6. 2021Gradient Inversion with Generative Image Priorcited by 101
  7. 2021DENSE: Data-Free One-Shot Federated Learningcited by 100
  8. 2022Trustworthy Graph Neural Networks: Aspects, Methods and Trendscited by 88
  9. 2023Federated Learning for Medical Image Analysis: A Surveycited by 77
  10. 2021CAFE: Catastrophic Data Leakage in Vertical Federated Learningcited by 76
  11. 2022Fishing for User Data in Large-Batch Federated Learning via Gradient Magnificationcited by 72
  12. 2022Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakagecited by 72
38 more
  1. 2022Recovering Private Text in Federated Learning of Language Modelscited by 67
  2. 2021The Skellam Mechanism for Differentially Private Federated Learningcited by 66
  3. 2021Eluding Secure Aggregation in Federated Learning via Model Inconsistencycited by 66
  4. 2022Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Modelscited by 52
  5. 2022LAMP: Extracting Text from Gradients with Language Model Priorscited by 52
  6. 2022GradViT: Gradient Inversion of Vision Transformerscited by 52
  7. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 48
  8. 2023GIFD: A Generative Gradient Inversion Method with Feature Domain Optimizationcited by 45
  9. 2022Do Gradient Inversion Attacks Make Federated Learning Unsafe?cited by 44
  10. 2023Bounding Training Data Reconstruction in DP-SGDcited by 42
  11. 2022Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentationcited by 41
  12. 2021Towards General Deep Leakage in Federated Learningcited by 40
  13. 2022Structural Pruning via Latency-Saliency Knapsackcited by 40
  14. 2023Recent Advances on Federated Learning: A Systematic Surveycited by 40
  15. 2022No Free Lunch Theorem for Security and Utility in Federated Learningcited by 39
  16. 2022A Survey on Gradient Inversion: Attacks, Defenses and Future Directionscited by 39
  17. 2022Gradient Obfuscation Gives a False Sense of Security in Federated Learningcited by 37
  18. 2022Topology-aware Generalization of Decentralized SGDcited by 37
  19. 2022On Privacy and Personalization in Cross-Silo Federated Learningcited by 36
  20. 2023Diffusion-based Conditional ECG Generation with Structured State Space Modelscited by 35
  21. 2023LOKI: Large-scale Data Reconstruction Attack against Federated Learning through Model Manipulationcited by 35
  22. 2021Bayesian Framework for Gradient Leakagecited by 34
  23. 2022Joint Privacy Enhancement and Quantization in Federated Learningcited by 34
  24. 2022Masked Jigsaw Puzzle: A Versatile Position Embedding for Vision Transformerscited by 33
  25. 2022Trading Off Privacy, Utility and Efficiency in Federated Learningcited by 33
  26. 2021Data-Free Knowledge Transfer: A Surveycited by 31
  27. 2022Learning to Invert: Simple Adaptive Attacks for Gradient Inversion in Federated Learningcited by 30
  28. 2022Cocktail Party Attack: Breaking Aggregation-Based Privacy in Federated Learning using Independent Component Analysiscited by 29
  29. 2024Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Modelscited by 28
  30. 2022Reconstructing Training Data from Model Gradient, Provablycited by 27
  31. 2024Enhancing One-Shot Federated Learning Through Data and Ensemble Co-Boostingcited by 27
  32. 2021Personalized Federated Learning with Adaptive Batchnorm for Healthcarecited by 26
  33. 2022Data Leakage in Federated Averagingcited by 26
  34. 2023Privacy and Fairness in Federated Learning: on the Perspective of Trade-offcited by 26
  35. 2022Sketching for First Order Method: Efficient Algorithm for Low-Bandwidth Channel and Vulnerabilitycited by 25
  36. 2022FRAug: Tackling Federated Learning with Non-IID Features via Representation Augmentationcited by 24
  37. 2022CANIFE: Crafting Canaries for Empirical Privacy Measurement in Federated Learningcited by 24
  38. 2023A Survey of Trustworthy Federated Learning with Perspectives on Security, Robustness, and Privacycited by 24

50 most cited of 320 All 320 (JSON)