mlsanityData to 7 Oct 2026

Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Models

arXiv 2201.12675 ↗First posted 29 Jan 2022cs.LG cs.CL cs.CRcited by 52

Not on the map. Why?

Abstract (arXiv)

A central tenet of Federated learning (FL), which trains models without centralizing user data, is privacy. However, previous work has shown that the gradient updates used in FL can leak user information. While the most industrial uses of FL are for text applications (e.g. keystroke prediction), nearly all attacks on FL privacy have focused on simple image classifiers. We propose a novel attack that reveals private user text by deploying malicious parameter vectors, and which succeeds even with mini-batches, multiple users, and long sequences. Unlike previous attacks on FL, the attack exploits characteristics of both the Transformer architecture and the token embedding, separately extracting tokens and positional embeddings to retrieve high-fidelity text. This work suggests that FL on text, which has historically been resistant to privacy attacks, is far more vulnerable than previously thought.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). Builds on 33 of its 68 references that are in the map. First posted on arXiv 29 Jan 2022.

Builds on

33 of 68 references.

References · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2016Pointer Sentinel Mixture Modelscited by 2,992
  6. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  7. 2020Extracting Training Data from Large Language Modelscited by 2,144
  8. 2019Deep Leakage from Gradientscited by 1,004
  9. 2019Towards Federated Learning at Scale: System Designcited by 826
  10. 2018LEAF: A Benchmark for Federated Settingscited by 780
  11. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  12. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
21 more
  1. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
  2. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  3. 2021See through Gradients: Image Batch Recovery via GradInversioncited by 320
  4. 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
  5. 2021A Field Guide to Federated Optimizationcited by 312
  6. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 310
  7. 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 242
  8. 2019Privacy-preserving Federated Brain Tumour Segmentationcited by 169
  9. 2018Protection Against Reconstruction and Its Applications in Private Federated Learningcited by 127
  10. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
  11. 2021When the Curious Abandon Honesty: Federated Learning Is Not Privatecited by 126
  12. 2019Federated Learning for Emoji Prediction in a Mobile Keyboardcited by 115
  13. 2021Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Modelscited by 106
  14. 2020When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?cited by 102
  15. 2020Residual Energy-Based Models for Text Generationcited by 95
  16. 2019Analyzing Information Leakage of Updates to Natural Language Modelscited by 89
  17. 2021Federated Evaluation and Tuning for On-Device Personalization: System Design & Applicationscited by 84
  18. 2021Differentially Private Federated Learning on Heterogeneous Datacited by 74
  19. 2021TAG: Gradient Attack on Transformer-based Language Modelscited by 63
  20. 2018Algorithms that Remember: Model Inversion Attacks and Data Protection Lawcited by 52
  21. 2018Tight Lower Bounds for Locally Differentially Private Selectioncited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 12; 2024: 15; 2025: 12; 2026: 5.

52 papers.

Sort:

By area:

  1. 2022Truth Serum: Poisoning Machine Learning Models to Reveal Their Secretscited by 94
  2. 2022Recovering Private Text in Federated Learning of Language Modelscited by 67
  3. 2022LAMP: Extracting Text from Gradients with Language Model Priorscited by 52
  4. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 48
  5. 2023FedML-HE: An Efficient Homomorphic-Encryption-Based Privacy-Preserving Federated Learning Systemcited by 38
  6. 2022Learning to Invert: Simple Adaptive Attacks for Gradient Inversion in Federated Learningcited by 30
  7. 2023Identifying and Mitigating Privacy Risks Stemming from Language Models: A Surveycited by 30
  8. 2023Integration of Large Language Models and Federated Learningcited by 28
  9. 2024Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Modelscited by 28
  10. 2024PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restorationcited by 27
  11. 2022Data Leakage in Federated Averagingcited by 26
  12. 2022CANIFE: Crafting Canaries for Empirical Privacy Measurement in Federated Learningcited by 24
38 more
  1. 2023Grounding Foundation Models through Federated Transfer Learning: A General Frameworkcited by 24
  2. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  3. 2024Federated Large Language Models: Current Progress and Future Directionscited by 22
  4. 2023Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learningcited by 21
  5. 2024Privacy Backdoors: Stealing Data with Corrupted Pretrained Modelscited by 20
  6. 2024SPEAR:Exact Gradient Inversion of Batches in Federated Learningcited by 14
  7. 2024DAGER: Exact Gradient Inversion for Large Language Modelscited by 12
  8. 2024Information Leakage from Embedding in Large Language Modelscited by 10
  9. 2023Hiding in Plain Sight: Disguising Data Stealing Attacks in Federated Learningcited by 9
  10. 2023Gradient-Free Privacy Leakage in Federated Language Models through Selective Weight Tamperingcited by 9
  11. 2023Beyond Gradient and Priors in Privacy Attacks: Leveraging Pooler Layer Inputs of Language Models in Federated Learningcited by 6
  12. 2024A Method to Facilitate Membership Inference Attacks in Deep Learning Modelscited by 6
  13. 2025Uncovering Gradient Inversion Risks in Practical Language Model Trainingcited by 6
  14. 2025GRAIN: Exact Graph Reconstruction from Gradientscited by 5
  15. 2025Privacy-Preserving LLM Embedding Transmission for End-Cloud Collaborationcited by 5
  16. 2025Selective Attention Federated Learning: Improving Privacy and Efficiency for Clinical Text Classificationcited by 5
  17. 2023BC4LLM: Trusted Artificial Intelligence When Blockchain Meets Large Language Modelscited by 4
  18. 2024Gradient Inversion Attack on Graph Neural Networkscited by 4
  19. 2024PPFedIT: Towards Privacy-Preserving Federated Instruction Tuning with Few-shot Local Examplescited by 3
  20. 2024Seeing the Forest through the Trees: Data Leakage from Partial Transformer Gradientscited by 3
  21. 2025Gradient Inversion Attacks on Parameter-Efficient Fine-Tuningcited by 3
  22. 2022Combining Stochastic Defenses to Resist Gradient Inversion: An Ablation Studycited by 2
  23. 2023Privacy Preserving Bayesian Federated Learning in Heterogeneous Settingscited by 2
  24. 2024Dealing Doubt: Unveiling Threat Models in Gradient Inversion Attacks under Federated Learning, A Survey and Taxonomycited by 2
  25. 2024Privacy-Preserving Data Deduplication for Enhancing Federated Learning of Language Models (Extended Version)cited by 2
  26. 2025Cutting Through Privacy: A Hyperplane-Based Data Reconstruction Attack in Federated Learningcited by 2
  27. 2026Mitigating Gradient Inversion Risks in Language Models via Token Obfuscationcited by 2
  28. 2025Empirical Calibration and Metric Differential Privacy in Language Modelscited by 1
  29. 2025Hear No Evil: Detecting Gradient Leakage by Malicious Servers in Federated Learningcited by 1
  30. 2026No More Guessing: a Verifiable Gradient Inversion Attack in Federated Learningcited by 1
  31. 2022Encrypted machine learning of molecular quantum propertiescited by 0
  32. 2023Neural Language Model Pruning for Automatic Speech Recognitioncited by 0
  33. 2024Privacy Attack in Federated Learning is Not Easy: An Experimental Studycited by 0
  34. 2025FedEM: A Privacy-Preserving Framework for Concurrent Utility Preservation in Federated Learningcited by 0
  35. 2025AnalogFed: Privacy-Preserving Discovery of Analog Circuits at Scale with Federated Generative AIcited by 0
  36. 2025Evaluating Selective Encryption Against Gradient Inversion Attackscited by 0
  37. 2025Zero-Shot Decentralized Federated Learningcited by 0
  38. 2026ARES: Scalable and Practical Gradient Inversion Attack in Federated Learning through Activation Recoverycited by 0

50 most cited of 52 All 52 (JSON)