mlsanityData to 14 Oct 2026

Seeing the Forest through the Trees: Data Leakage from Partial Transformer Gradients

arXiv 2406.00999 ↗First posted 3 Jun 2024cs.LG cs.CL cs.CRcited by 3

Not on the map. Why?

Abstract (arXiv)

Recent studies have shown that distributed machine learning is vulnerable to gradient inversion attacks, where private training data can be reconstructed by analyzing the gradients of the models shared in training. Previous attacks established that such reconstructions are possible using gradients from all parameters in the entire models. However, we hypothesize that most of the involved modules, or even their sub-modules, are at risk of training data leakage, and we validate such vulnerabilities in various intermediate layers of language models. Our extensive experiments reveal that gradients from a single Transformer layer, or even a single linear component with 0.54% parameters, are susceptible to training data leakage. Additionally, we show that applying differential privacy on gradients during training offers limited protection against the novel vulnerability of data disclosure.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 27 of its 44 references that are in the map. First posted on arXiv 3 Jun 2024.

Builds on

27 of 44 references.

References · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2016Deep Learning with Differential Privacycited by 2,728
  4. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  5. 2019Deep Leakage from Gradientscited by 1,021
  6. 2018Neural Network Acceptability Judgmentscited by 958
  7. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
  8. 2021Datasets: A Community Library for Natural Language Processingcited by 530
  9. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 383
  10. 2020iDLG: Improved Deep Leakage from Gradientscited by 382
  11. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 234
  12. 2019A Survey on Distributed Machine Learningcited by 188
15 more
  1. 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
  2. 2020Provable Defense against Privacy Leakage in Federated Learning from Representation Perspectivecited by 94
  3. 2022Recovering Private Text in Federated Learning of Language Modelscited by 70
  4. 2021TAG: Gradient Attack on Transformer-based Language Modelscited by 63
  5. 2022LAMP: Extracting Text from Gradients with Language Model Priorscited by 54
  6. 2022Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Modelscited by 53
  7. 2021Layer-wise Adaptive Model Aggregation for Scalable Federated Learningcited by 36
  8. 2021Bayesian Framework for Gradient Leakagecited by 34
  9. 2020Federated Transfer Learning: concept and applicationscited by 32
  10. 2022Learning to Invert: Simple Adaptive Attacks for Gradient Inversion in Federated Learningcited by 30
  11. 2022Reconstructing Training Data from Model Gradient, Provablycited by 27
  12. 2021Revealing and Protecting Labels in Distributed Trainingcited by 17
  13. 2024SoK: On Gradient Leakage in Federated Learningcited by 12
  14. 2020Scalable Privacy-Preserving Distributed Learningcited by 9
  15. 2023Beyond Gradient and Priors in Privacy Attacks: Leveraging Pooler Layer Inputs of Language Models in Federated Learningcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Empirical Calibration and Metric Differential Privacy in Language Modelscited by 1
  2. 2025PC-MoE: Memory-Efficient and Privacy-Preserving Collaborative Training for Mixture-of-Experts LLMscited by 1
  3. 2026In Transformer We Trust? A Perspective on Transformer Architecture Failure Modescited by 1