# LAMP: Extracting Text from Gradients with Language Model Priors

arXiv: https://arxiv.org/abs/2202.08827 · HTML: https://mlsanity.com/paper/2202.08827/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 53 references that are in the map.
- First posted on arXiv 17 Feb 2022.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [TinyBERT: Distilling BERT for Natural Language Understanding](https://mlsanity.com/paper/1909.10351/) (2019)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [Inverting Gradients -- How easy is it to break privacy in federated learning?](https://mlsanity.com/paper/2003.14053/) (2020)
- [Federated Optimization: Distributed Machine Learning for On-Device Intelligence](https://mlsanity.com/paper/1610.02527/) (2016)
- [iDLG: Improved Deep Leakage from Gradients](https://mlsanity.com/paper/2001.02610/) (2020)
- [See through Gradients: Image Batch Recovery via GradInversion](https://mlsanity.com/paper/2104.07586/) (2021)
- [Gradient-based Adversarial Attacks against Text Transformers](https://mlsanity.com/paper/2104.13733/) (2021)
- [Evaluating Gradient Inversion Attacks and Defenses in Federated Learning](https://mlsanity.com/paper/2112.00059/) (2021)
- [Deep Learning Based Text Classification: A Comprehensive Review](https://mlsanity.com/paper/2004.03705/) (2020)
- [Federated Learning for Emoji Prediction in a Mobile Keyboard](https://mlsanity.com/paper/1906.04329/) (2019)
- [Gradient Inversion with Generative Image Prior](https://mlsanity.com/paper/2110.14962/) (2021)
- [R-GAP: Recursive Gradient Attack on Privacy](https://mlsanity.com/paper/2010.07733/) (2020)
- [Provable Defense against Privacy Leakage in Federated Learning from Representation Perspective](https://mlsanity.com/paper/2012.06043/) (2020)
- [A Framework for Evaluating Gradient Leakage Attacks in Federated Learning](https://mlsanity.com/paper/2004.10397/) (2020)
- [TAG: Gradient Attack on Transformer-based Language Models](https://mlsanity.com/paper/2103.06819/) (2021)
- [Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Models](https://mlsanity.com/paper/2201.12675/) (2022)
- [Towards General Deep Leakage in Federated Learning](https://mlsanity.com/paper/2110.09074/) (2021)
- [Bayesian Framework for Gradient Leakage](https://mlsanity.com/paper/2111.04706/) (2021)
- [PRECODE - A Generic Model Extension to Prevent Deep Gradient Leakage](https://mlsanity.com/paper/2108.04725/) (2021)

## Built on by (most cited first, 30 of 54)

- [Security and Privacy Challenges of Large Language Models: A Survey](https://mlsanity.com/paper/2402.00888/) (2024)
- [Towards Building the Federated GPT: Federated Instruction Tuning](https://mlsanity.com/paper/2305.05644/) (2023)
- [Recovering Private Text in Federated Learning of Language Models](https://mlsanity.com/paper/2205.08514/) (2022)
- [Locally Differentially Private Document Generation Using Zero Shot Prompting](https://mlsanity.com/paper/2310.16111/) (2023)
- [Privacy in Large Language Models: Attacks, Defenses and Future Directions](https://mlsanity.com/paper/2310.10383/) (2023)
- [Privacy Implications of Retrieval-Based Language Models](https://mlsanity.com/paper/2305.14888/) (2023)
- [Integration of Large Language Models and Federated Learning](https://mlsanity.com/paper/2307.08925/) (2023)
- [Identifying and Mitigating Privacy Risks Stemming from Language Models: A Survey](https://mlsanity.com/paper/2310.01424/) (2023)
- [Data Leakage in Federated Averaging](https://mlsanity.com/paper/2206.12395/) (2022)
- [FwdLLM: Efficient FedLLM using Forward Gradient](https://mlsanity.com/paper/2308.13894/) (2023)
- [Grounding Foundation Models through Federated Transfer Learning: A General Framework](https://mlsanity.com/paper/2311.17431/) (2023)
- [Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Survey](https://mlsanity.com/paper/2406.05392/) (2024)
- [A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiency](https://mlsanity.com/paper/2307.10655/) (2023)
- [AI Risk Management Should Incorporate Both Safety and Security](https://mlsanity.com/paper/2405.19524/) (2024)
- [Federated Few-Shot Learning for Mobile NLP](https://mlsanity.com/paper/2212.05974/) (2022)
- [SPEAR:Exact Gradient Inversion of Batches in Federated Learning](https://mlsanity.com/paper/2403.03945/) (2024)
- [Trustworthy Distributed AI Systems: Robustness, Privacy, and Governance](https://mlsanity.com/paper/2402.01096/) (2024)
- [DAGER: Exact Gradient Inversion for Large Language Models](https://mlsanity.com/paper/2405.15586/) (2024)
- [TabLeak: Tabular Data Leakage in Federated Learning](https://mlsanity.com/paper/2210.01785/) (2022)
- [Inf2Guard: An Information-Theoretic Framework for Learning Privacy-Preserving Representations against Inference Attacks](https://mlsanity.com/paper/2403.02116/) (2024)
- [Textual Unlearning Gives a False Sense of Unlearning](https://mlsanity.com/paper/2406.13348/) (2024)
- [Information Leakage from Embedding in Large Language Models](https://mlsanity.com/paper/2405.11916/) (2024)
- [Hiding in Plain Sight: Disguising Data Stealing Attacks in Federated Learning](https://mlsanity.com/paper/2306.03013/) (2023)
- [LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasures](https://mlsanity.com/paper/2505.01177/) (2025)
- [Uncovering Gradient Inversion Risks in Practical Language Model Training](https://mlsanity.com/paper/2507.21198/) (2025)
- [Beyond Gradient and Priors in Privacy Attacks: Leveraging Pooler Layer Inputs of Language Models in Federated Learning](https://mlsanity.com/paper/2312.05720/) (2023)
- [GRAIN: Exact Graph Reconstruction from Gradients](https://mlsanity.com/paper/2503.01838/) (2025)
- [FedEval-LLM: Federated Evaluation of Large Language Models on Downstream Tasks with Collective Wisdom](https://mlsanity.com/paper/2404.12273/) (2024)
- [Seeing the Forest through the Trees: Data Leakage from Partial Transformer Gradients](https://mlsanity.com/paper/2406.00999/) (2024)
- [MIP: CLIP-based Image Reconstruction from PEFT Gradients](https://mlsanity.com/paper/2403.07901/) (2024)

## Abstract (arXiv)

Recent work shows that sensitive user data can be reconstructed from gradient updates, breaking the key privacy promise of federated learning. While success was demonstrated primarily on image data, these methods do not directly transfer to other domains such as text. In this work, we propose LAMP, a novel attack tailored to textual data, that successfully reconstructs original text from gradients. Our attack is based on two key insights: (i) modeling prior text probability with an auxiliary language model, guiding the search towards more natural text, and (ii) alternating continuous and discrete optimization, which minimizes reconstruction loss on embeddings, while avoiding local minima by applying discrete text transformations. Our experiments demonstrate that LAMP is significantly more effective than prior work: it reconstructs 5x more bigrams and 23% longer subsequences on average. Moreover, we are the first to recover inputs from batch sizes larger than 1 for textual models. These findings indicate that gradient updates of models operating on textual data leak more information than previously thought.