mlsanityData to 14 Oct 2026

Reconstructing Training Data from Model Gradient, Provably

arXiv 2212.03714 ↗First posted 7 Dec 2022cs.LG cs.CR stat.MLcited by 27

Not on the map. Why?

Abstract (arXiv)

Understanding when and how much a model gradient leaks information about the training sample is an important question in privacy. In this paper, we present a surprising result: even without training or memorizing the data, we can fully reconstruct the training samples from a single gradient query at a randomly chosen parameter value. We prove the identifiability of the training data under mild conditions: with shallow or deep neural networks and a wide range of activation functions. We also present a statistically and computationally efficient algorithm based on tensor decomposition to reconstruct the training data. As a provable attack that reveals sensitive training data, our findings suggest potential severe threats to privacy, especially in federated learning.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 31 of its 49 references that are in the map. First posted on arXiv 7 Dec 2022.

Builds on

31 of 49 references.

References · 31

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  3. 2018Federated Learning with Non-IID Datacited by 1,108
  4. 2019Deep Leakage from Gradientscited by 1,021
  5. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  6. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
  7. 2017Federated Multi-Task Learningcited by 672
  8. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 574
  9. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 564
  10. 2020iDLG: Improved Deep Leakage from Gradientscited by 382
  11. 2021See through Gradients: Image Batch Recovery via GradInversioncited by 326
  12. 2021Evaluating Gradient Inversion Attacks and Defenses in Federated Learningcited by 196
19 more
  1. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 195
  2. 2019Private Stochastic Convex Optimization with Optimal Ratescited by 186
  3. 2014Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Boundscited by 107
  4. 2021Gradient Inversion with Generative Image Priorcited by 101
  5. 2020R-GAP: Recursive Gradient Attack on Privacycited by 97
  6. 2017Recovery Guarantees for One-hidden-layer Neural Networkscited by 94
  7. 2020Provable Defense against Privacy Leakage in Federated Learning from Representation Perspectivecited by 94
  8. 2022Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Losscited by 61
  9. 2022GradViT: Gradient Inversion of Vision Transformerscited by 53
  10. 2021Bayesian Framework for Gradient Leakagecited by 34
  11. 2019Inverting Deep Generative models, One layer at a timecited by 33
  12. 2020Rethinking Privacy Preserving Deep Learning: How to Evaluate and Thwart Privacy Attackscited by 32
  13. 2020SAPAG: A Self-Adaptive Privacy Attack From Gradientscited by 27
  14. 2014Guaranteed Non-Orthogonal Tensor Decomposition via Alternating Rank-1 Updatescited by 21
  15. 2015Fast and Guaranteed Tensor Decomposition via Sketchingcited by 19
  16. 2021Understanding Training-Data Leakage from Gradients in Neural Networks for Image Classificationcited by 16
  17. 2015Tensor Factorization via Matrix Factorizationcited by 15
  18. 2013Uniqueness of Tensor Decompositions with Applications to Polynomial Identifiabilitycited by 9
  19. 2016Online and Differentially-Private Tensor Decompositioncited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 10; 2025: 8; 2026: 1.

27 papers.

Sort:

By area:

  1. 2023Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Lossescited by 26
  2. 2024Reconstruction Attacks on Machine Unlearning: Simple Models are Vulnerablecited by 25
  3. 2023Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learningcited by 21
  4. 2023Decentralized SGD and Average-direction SAM are Asymptotically Equivalentcited by 15
  5. 2024Inf2Guard: An Information-Theoretic Framework for Learning Privacy-Preserving Representations against Inference Attackscited by 13
  6. 2023Unmasking Transformers: A Theoretical Approach to Data Recovery via Attention Weightscited by 12
  7. 2023Collaborative Learning via Prediction Consensuscited by 8
  8. 2023Understanding Deep Gradient Leakage via Inversion Influence Functionscited by 6
  9. 2023Beyond Gradient and Priors in Privacy Attacks: Leveraging Pooler Layer Inputs of Language Models in Federated Learningcited by 6
  10. 2024Data Reconstruction Attacks and Defenses: A Systematic Evaluationcited by 6
  11. 2023Optimizing Orthogonalized Tensor Deflation via Random Tensor Theorycited by 4
  12. 2024Gradient Inversion Attack on Graph Neural Networkscited by 4
15 more
  1. 2024Seeing the Forest through the Trees: Data Leakage from Partial Transformer Gradientscited by 3
  2. 2025A Law of Data Reconstruction for Random Features (and Beyond)cited by 3
  3. 2024Network Inversion for Training-Like Data Reconstructioncited by 1
  4. 2025Online Learning and Unlearningcited by 1
  5. 2025Gradient Inversion Transcript: Leveraging Robust Generative Priors to Reconstruct Training Data from Gradient Leakagecited by 1
  6. 2025Efficient Unlearning with Privacy Guaranteescited by 1
  7. 2024Data Quality in Edge Machine Learning: A State-of-the-Art Surveycited by 0
  8. 2024R-CONV: An Analytical Approach for Efficient Data Reconstruction via Convolutional Gradientscited by 0
  9. 2024Optimal Defenses Against Gradient Reconstruction Attackscited by 0
  10. 2024Network Inversion and Its Applicationscited by 0
  11. 2025Training Set Reconstruction from Differentially Private Forests: How Effective is DP?cited by 0
  12. 2025Personalized Federated Training of Diffusion Models with Privacy Guaranteescited by 0
  13. 2025DRAUN: An Algorithm-Agnostic Data Reconstruction Attack on Federated Unlearning Systemscited by 0
  14. 2025GraphToxin: Reconstructing Full Unlearned Graphs from Graph Unlearningcited by 0
  15. 2026Efficient Techniques for Data Reconstruction, with Finite-Width Recovery Guaranteescited by 0