# FISGuard: Defending Against Membership Inference via Fixed Input Subspaces

arXiv: https://arxiv.org/abs/2608.27836 · HTML: https://mlsanity.com/paper/2608.27836/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security.
- Builds on 18 of its 30 references that are in the map.
- First posted on arXiv 28 Aug 2026.

## Placed in

- LLM safety and security / [Federated unlearning and poisoning](https://mlsanity.com/safety/federated-unlearning-and-poisoning/)

## Builds on (most cited first, 18 of 18)

- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [Machine Learning with Membership Privacy using Adversarial Regularization](https://mlsanity.com/paper/1807.05852/) (2018)
- [Practical Secure Aggregation for Federated Learning on User-Held Data](https://mlsanity.com/paper/1611.04482/) (2016)
- [FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning](https://mlsanity.com/paper/2309.00363/) (2023)
- [Understanding Gradient Clipping in Private SGD: A Geometric Perspective](https://mlsanity.com/paper/2006.15429/) (2020)
- [Local and Central Differential Privacy for Robustness and Privacy in Federated Learning](https://mlsanity.com/paper/2009.03561/) (2020)
- [Towards Label-Only Membership Inference Attack against Pre-trained Large Language Models](https://mlsanity.com/paper/2502.18943/) (2025)
- [Scaling Federated Learning for Fine-tuning of Large Language Models](https://mlsanity.com/paper/2102.00875/) (2021)
- [Privacy Regularization: Joint Privacy-Utility Optimization in Language Models](https://mlsanity.com/paper/2103.07567/) (2021)
- [An Accuracy-Lossless Perturbation Method for Defending Privacy Attacks in Federated Learning](https://mlsanity.com/paper/2002.09843/) (2020)
- [Federated pretraining and fine tuning of BERT using clinical notes from multiple silos](https://mlsanity.com/paper/2002.08562/) (2020)
- [Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions](https://mlsanity.com/paper/2412.16504/) (2024)
- [MIAShield: Defending Membership Inference Attacks via Preemptive Exclusion of Members](https://mlsanity.com/paper/2203.00915/) (2022)
- [Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacy](https://mlsanity.com/paper/2305.00873/) (2023)
- [Little is Enough: Boosting Privacy by Sharing Only Hard Labels in Federated Semi-Supervised Learning](https://mlsanity.com/paper/2310.05696/) (2023)
- [Toward Efficient Membership Inference Attacks against Federated Large Language Models: A Projection Residual Approach](https://mlsanity.com/paper/2604.21197/) (2026)
- [United We Defend: Collaborative Membership Inference Defenses in Federated Learning](https://mlsanity.com/paper/2601.06866/) (2026)

## Abstract (arXiv)

As large language models are increasingly adopted in federated learning, protecting user privacy while performing parameter-efficient fine-tuning on distributed private data has become an important challenge. Although clients only share gradients instead of directly uploading raw data, the shared gradients may still leak membership information about training samples. ProjRes (S&P, 2026) further increases this risk: with less information and without accessing model outputs, an attacker can effectively distinguish members from non-members solely based on the projection residual between a candidate representation and the subspace induced by server-observable gradients. Existing defenses against membership inference mostly rely on gradient perturbation or regularization, which can not only degrade model utility but also fail to effectively defend against the membership inference attack introduced by ProjRes, which exploits the geometric structure of gradients. To address this issue, we propose FISGuard, a lightweight defense. Its key idea is to construct and fix a low-dimensional representation subspace using independent public data, thereby restricting the space through which private representations are exposed via gradients while preserving the primary information required for downstream tasks. This substantially reduces the projection-residual discrepancy between members and non-members. We evaluate FISGuard against five representative defense methods across three NLP datasets, two LLMs, and two fine-tuning strategies, Adapter and LoRA. The results show that FISGuard reduces the ProjRes attack AUC to near the random-guessing level of 0.5 in most settings, while maintaining downstream task performance close to that of the undefended model and introducing only limited computational overhead, thereby achieving a favorable privacy--utility trade-off.