mlsanityData to 14 Oct 2026

Revisiting Gradient Pruning: A Dual Realization for Defending against Gradient Attacks

arXiv 2401.16687 ↗First posted 30 Jan 2024cs.CR cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Collaborative learning (CL) is a distributed learning framework that aims to protect user privacy by allowing users to jointly train a model by sharing their gradient updates only. However, gradient inversion attacks (GIAs), which recover users' training data from shared gradients, impose severe privacy threats to CL. Existing defense methods adopt different techniques, e.g., differential privacy, cryptography, and perturbation defenses, to defend against the GIAs. Nevertheless, all current defense methods suffer from a poor trade-off between privacy, utility, and efficiency. To mitigate the weaknesses of existing solutions, we propose a novel defense method, Dual Gradient Pruning (DGP), based on gradient pruning, which can improve communication efficiency while preserving the utility and privacy of CL. Specifically, DGP slightly changes gradient pruning with a stronger privacy guarantee. And DGP can also significantly improve communication efficiency with a theoretical analysis of its convergence and generalization. Our extensive experiments show that DGP can effectively defend against the most powerful GIAs and reduce the communication cost without sacrificing the model's utility.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 36 of its 50 references that are in the map. First posted on arXiv 30 Jan 2024.

Builds on

36 of 50 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2019Deep Leakage from Gradientscited by 1,021
  6. 2016Deep Variational Information Bottleneckcited by 930
  7. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
  8. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 536
  9. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 478
  10. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 459
  11. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
  12. 2020iDLG: Improved Deep Leakage from Gradientscited by 382
24 more
  1. 2021See through Gradients: Image Batch Recovery via GradInversioncited by 326
  2. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 310
  3. 2017The Marginal Value of Adaptive Gradient Methods in Machine Learningcited by 250
  4. 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 246
  5. 2021Evaluating Gradient Inversion Attacks and Defenses in Federated Learningcited by 196
  6. 2017Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryptioncited by 176
  7. 2020Understanding Gradient Clipping in Private SGD: A Geometric Perspectivecited by 142
  8. 2021When the Curious Abandon Honesty: Federated Learning Is Not Privatecited by 127
  9. 2021Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Modelscited by 108
  10. 2021Gradient Inversion with Generative Image Priorcited by 101
  11. 2020R-GAP: Recursive Gradient Attack on Privacycited by 97
  12. 2020Provable Defense against Privacy Leakage in Federated Learning from Representation Perspectivecited by 94
  13. 2022Fishing for User Data in Large-Batch Federated Learning via Gradient Magnificationcited by 73
  14. 2022Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakagecited by 72
  15. 2019Differentially Private Model Publishing for Deep Learningcited by 71
  16. 2020A Framework for Evaluating Gradient Leakage Attacks in Federated Learningcited by 64
  17. 2020LDP-FL: Practical Private Aggregation in Federated Learning with Local Differential Privacycited by 64
  18. 2020Rethinking Privacy Preserving Deep Learning: How to Evaluate and Thwart Privacy Attackscited by 32
  19. 2020Privacy-preserving Collaborative Learning with Automatic Transformation Searchcited by 31
  20. 2020SAPAG: A Self-Adaptive Privacy Attack From Gradientscited by 27
  21. 2020FLAME: Differentially Private Federated Learning in the Shuffle Modelcited by 27
  22. 2021ScaleCom: Scalable Sparsified Gradient Compression for Communication-Efficient Distributed Trainingcited by 22
  23. 2019Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learningcited by 18
  24. 2020Exploring the Security Boundary of Data Reconstruction via Neuron Exclusivity Analysiscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 3.

7 papers.

  1. 2024A Survey of Privacy-Preserving Model Explanations: Privacy Risks, Attacks, and Countermeasurescited by 6
  2. 2025SVDefense: Effective Defense against Gradient Inversion Attacks via Singular Value Decompositioncited by 2
  3. 2026Reconstructing Training Data from Adapter-based Federated Large Language Modelscited by 1
  4. 2025Images in Motion?: A First Look into Video Leakage in Collaborative Deep Learningcited by 0
  5. 2025Towards Privacy-Preserving and Heterogeneity-aware Split Federated Learning via Probabilistic Maskingcited by 0
  6. 2026UnlearnShield: Shielding Forgotten Privacy against Unlearning Inversioncited by 0
  7. 2026ARES: Scalable and Practical Gradient Inversion Attack in Federated Learning through Activation Recoverycited by 0