mlsanityData to 14 Oct 2026

Toward Efficient Membership Inference Attacks against Federated Large Language Models: A Projection Residual Approach

arXiv 2604.21197 ↗First posted 23 Apr 2026cs.LGcited by 3

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Federated Large Language Models (FedLLMs) enable multiple parties to collaboratively fine-tune LLMs without sharing raw data, addressing challenges of limited resources and privacy concerns. Despite data localization, shared gradients can still expose sensitive information through membership inference attacks (MIAs). However, FedLLMs' unique properties, i.e. massive parameter scales, rapid convergence, and sparse, non-orthogonal gradients, render existing MIAs ineffective. To address this gap, we propose ProjRes, the first projection residuals-based passive MIA tailored for FedLLMs. ProjRes leverages hidden embedding vectors as sample representations and analyzes their projection residuals on the gradient subspace to uncover the intrinsic link between gradients and inputs. It requires no shadow models, auxiliary classifiers, or historical updates, ensuring efficiency and robustness. Experiments on four benchmarks and four LLMs show that ProjRes achieves near 100% accuracy, outperforming prior methods by up to 75.75%, and remains effective even under strong differential privacy defenses. Our findings reveal a previously overlooked privacy vulnerability in FedLLMs and call for a re-examination of their security assumptions. Our code and data are available at hrefhttps://anonymous.4open.science/r/Passive-MIA-5268link.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 48 of its 75 references that are in the map. First posted on arXiv 23 Apr 2026.

Builds on

48 of 75 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Cascading and Proxy Membership Inference Attacks29 Jul
  2. 2025Towards Label-Only Membership Inference Attack against Pre-trained Large Language Models26 Feb
  3. 2024Defending Membership Inference Attacks via Privacy-aware Sparsity Tuning9 Oct
  4. 2024Is Difficulty Calibration All We Need? Towards More Practical Membership Inference Attacks31 Aug
  5. 2024SeqMIA: Sequential-Metric Based Membership Inference Attack21 Jul
  6. 2024Learning-Based Difficulty Calibration for Enhanced Membership Inference Attacks10 Jan
  7. 2022Membership Inference Attacks by Exploiting Loss Trajectory31 Aug
  8. 2022How to Combine Membership-Inference Attacks on Multiple Updated Models12 May
  9. 2021Accuracy-Privacy Trade-off in Deep Ensemble: A Membership Inference Perspective12 May

Other references · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  5. 2024Qwen2 Technical Reportcited by 2,120
  6. 2019Deep Leakage from Gradientscited by 1,021
  7. 2018Neural Network Acceptability Judgmentscited by 958
  8. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 788
  9. 2025Qwen2.5-Omni Technical Reportcited by 736
  10. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 520
  11. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 383
  12. 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 292
27 more
  1. 2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Modelscited by 155
  2. 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 149
  3. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 128
  4. 2024Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resourcescited by 93
  5. 2016Yelp Dataset Challenge: Review Rating Predictioncited by 84
  6. 2023Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimizationcited by 65
  7. 2020A Framework for Evaluating Gradient Leakage Attacks in Federated Learningcited by 64
  8. 2023FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learningcited by 60
  9. 2024FDLoRA: Personalized Federated Learning of Large Language Model via Dual LoRA Tuningcited by 59
  10. 2024Dual-Personalizing Adapter for Federated Foundation Modelscited by 55
  11. 2023Prompt Federated Learning for Weather Forecasting: Toward Foundation Models on Meteorological Datacited by 40
  12. 2024Membership Inference Attacks and Defenses in Federated Learning: A Surveycited by 31
  13. 2024The Future of Large Language Model Pre-training is Federatedcited by 30
  14. 2023Low-Parameter Federated Learning with Large Language Modelscited by 23
  15. 2022Federated Few-Shot Learning for Mobile NLPcited by 15
  16. 2024DAGER: Exact Gradient Inversion for Large Language Modelscited by 14
  17. 2024FedMIA: An Effective Membership Inference Attack Exploiting "All for One" Principle in Federated Learningcited by 12
  18. 2024Safely Learning with Private Data: A Federated Learning Framework for Large Language Modelcited by 11
  19. 2024Analysis of Privacy Leakage in Federated Large Language Modelscited by 9
  20. 2024DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptationcited by 9
  21. 2022Online Model Compression for Federated Learning with Large Modelscited by 7
  22. 2025Uncovering Gradient Inversion Risks in Practical Language Model Trainingcited by 7
  23. 2022Towards Practical Few-shot Federated NLPcited by 5
  24. 2024FedRDMA: Communication-Efficient Cross-Silo Federated LLM via Chunked RDMA Transmissioncited by 5
  25. 2022Towards Sparsified Federated Neuroimaging Models via Weight Pruningcited by 4
  26. 2024Save It All: Enabling Full Parameter Tuning for Federated Large Language Models via Cycle Block Gradient Descentcited by 4
  27. 2022Perfectly Accurate Membership Inference by a Dishonest Central Server in Federated Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026VLALeaks: Membership Inference Attacks against Vision-Language-Action Modelscited by 2
  2. 2026FISGuard: Defending Against Membership Inference via Fixed Input Subspacescited by 0
  3. 2026Calibrating One-Round Membership Inference with Neighborscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SafeLM: Unified Privacy-Aware Optimization for Trustworthy Federated Large Language Models
  2. 2024Privacy-Preserving Large Language Models: Mechanisms, Applications, and Future Directions
  3. 2023Gradient-Free Privacy Leakage in Federated Language Models through Selective Weight Tampering
  4. 2024Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security
  5. 2024Order of Magnitude Speedups for LLM Membership Inference
  6. 2025Membership Inference Attacks on Large-Scale Models: A Survey
  7. 2025SOFT: Selective Data Obfuscation for Protecting LLM Fine-tuning against Membership Inference Attacks
  8. 2024Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions