mlsanityData to 14 Oct 2026

TIGER: Inverting Transformer Gradients via Embedding-Subspace Distance Optimization

arXiv 2606.18312 ↗First posted 16 Jun 2026cs.CR cs.DC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated learning allows multiple clients to jointly train a shared model by sending gradient updates to a central server while keeping raw inputs local. However, prior gradient inversion attacks show that these updates can reveal enough information to reconstruct client inputs. Existing attacks on transformers either optimize dummy inputs to match the true client updates, which is costly and unstable for modern models, or exploit the low rank of attention gradients to identify a subspace containing the true layer embeddings, followed by a discrete membership test for candidate tokens. However, this token test is brittle under numerical noise, i.e., from quantization or Differential Privacy (DP), and scales poorly for encoder models with non-causal attention. We introduce TIGER, a continuous gradient inversion attack that turns this subspace signal into a differentiable objective. Instead of searching over tokens or matching full gradients, TIGER directly optimizes token embeddings to minimize their distance to the subspace. Our experiments demonstrate that on encoder-only models, TIGER substantially improves both reconstruction quality and runtime over existing attacks, while on decoder models, TIGER is more robust than prior subspace-based attacks, enabling the first successful reconstructions in DP-defended federated learning settings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 41 of its 53 references that are in the map. First posted on arXiv 16 Jun 2026.

Builds on

41 of 53 references.

References · 41

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2016Pointer Sentinel Mixture Modelscited by 3,096
  3. 2016Deep Learning with Differential Privacycited by 2,728
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  5. 2019Deep Leakage from Gradientscited by 1,021
  6. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
  7. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 459
  8. 2025MedGemma Technical Reportcited by 436
  9. 2020iDLG: Improved Deep Leakage from Gradientscited by 382
  10. 2021See through Gradients: Image Batch Recovery via GradInversioncited by 326
  11. 2019Differentially Private Learning with Adaptive Clippingcited by 220
  12. 2025EmbeddingGemma: Powerful and Lightweight Text Representationscited by 139
29 more
  1. 2021Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Modelscited by 108
  2. 2021Gradient Inversion with Generative Image Priorcited by 101
  3. 2020R-GAP: Recursive Gradient Attack on Privacycited by 97
  4. 2022Fishing for User Data in Large-Batch Federated Learning via Gradient Magnificationcited by 73
  5. 2022Recovering Private Text in Federated Learning of Language Modelscited by 70
  6. 2021TAG: Gradient Attack on Transformer-based Language Modelscited by 63
  7. 2023FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Modelscited by 63
  8. 2020Differentially Private Secure Multi-Party Computation for Federated Learning in Financial Applicationscited by 58
  9. 2022LAMP: Extracting Text from Gradients with Language Model Priorscited by 54
  10. 2022Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Modelscited by 53
  11. 2022GradViT: Gradient Inversion of Vision Transformerscited by 53
  12. 2026MedGemma 1.5 Technical Reportcited by 45
  13. 2021Towards General Deep Leakage in Federated Learningcited by 40
  14. 2023Integration of Large Language Models and Federated Learningcited by 31
  15. 2022Cocktail Party Attack: Breaking Aggregation-Based Privacy in Federated Learning using Independent Component Analysiscited by 29
  16. 2024Federated Large Language Models: Current Progress and Future Directionscited by 27
  17. 2022Differentially Private Learning with Per-Sample Adaptive Clippingcited by 21
  18. 2024SPEAR:Exact Gradient Inversion of Batches in Federated Learningcited by 15
  19. 2024DAGER: Exact Gradient Inversion for Large Language Modelscited by 14
  20. 2022Federated Multilingual Models for Medical Transcript Analysiscited by 11
  21. 2025FlowerTune: A Cross-Domain Benchmark for Federated Fine-Tuning of Large Language Modelscited by 11
  22. 2023Beyond Gradient and Priors in Privacy Attacks: Leveraging Pooler Layer Inputs of Language Models in Federated Learningcited by 6
  23. 2023FedJudge: Federated Legal Large Language Modelcited by 5
  24. 2025GRAIN: Exact Graph Reconstruction from Gradientscited by 5
  25. 2025Federated Large Language Models: Feasibility, Robustness, Security and Future Directionscited by 3
  26. 2025SPEAR++: Scaling Gradient Inversion via Sparsely-Used Dictionary Learningcited by 3
  27. 2025Federated Learning with Layer Skipping: Efficient Training of Large Language Models for Healthcare NLPcited by 2
  28. 2026Toward Federated Large Language Models in Medicine: A Parameter-Efficient Framework for Privacy-Preserving, Multi-Institutional Adaptationcited by 1
  29. 2026How Small Can You Go? LoRA Fine-Tuning 270M-8B Models for Merchant Information Extraction in Financial Transactionscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025GUIDE: Enhancing Gradient Inversion Attacks in Federated Learning with Denoising Models
  2. 2023Subspace based Federated Unlearning
  3. 2025Gradient Inversion in Federated Reinforcement Learning
  4. 2024Gradient Inversion of Federated Diffusion Models
  5. 2022Gradient Obfuscation Gives a False Sense of Security in Federated Learning
  6. 2025Enhanced Privacy Leakage from Noise-Perturbed Gradients via Gradient-Guided Conditional Diffusion Models
  7. 2024Geminio: Language-Guided Gradient Inversion Attacks in Federated Learning
  8. 2024Federated Learning Nodes Can Reconstruct Peers' Image Data