FedAttr: Towards Privacy-preserving Client-Level Attribution in Federated LLM Fine-tuning

arXiv 2605.06596 ↗First posted 7 May 2026cs.CR cs.LGcited by 0

On the map: LLM safety and security → Federated unlearning and poisoning

Abstract (arXiv)

Watermark radioactivity testing type of methods can detect whether a model was trained on watermarked documents, and have become key tools for protecting data ownership in the fine-tuning of large language models (LLMs). Existing works have proved their effectiveness in centralized LLM fine-tuning. However, this type of method faces several challenges and remains underexplored in federated learning (FL), a widely-applied paradigm for fine-tuning LLMs collaboratively on private data across different users. FL mainly ensures privacy through secure aggregation (SA), which allows the server to aggregate updates while keeping clients' updates private. This mechanism preserves privacy but makes it difficult to identify which client trained on watermarked documents. In this work, we propose FedAttr, a new client-level attribution protocol for FL. FedAttr identifies which clients trained on watermarked data via a paired-subset-difference mechanism, while preserving the privacy guarantees of SA and FL performance. FedAttr proceeds in three steps: (i) estimate each client's update by differencing two SA queries, (ii) score the estimate with the watermark detector via differential scoring, and (iii) combine scores across rounds via Stouffer method. We theoretically show that FedAttr produces an unbiased estimator of each client's update with bounded mutual information leakage (i.e., O(d^*/N) per-round update). Moreover, FedAttr empirically achieves 100% TPR and 0% FPR, outperforming all baselines by at least 44.4% in TPR or 19.1% in FPR, with only 6.3% overhead relative to FL training time. Ablation studies confirm that FedAttr is robust to protocol parameters and configurations.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security. Builds on 15 of its 21 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

15 of 21 references.

References · 15

2024The Llama 3 Herd of Modelscited by 14,486
2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,653
2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 833
2023A Watermark for Large Language Modelscited by 717
2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 185
2024FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptationscited by 145
2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 126
2022FLDetector: Defending Federated Learning Against Model Poisoning Attacks via Detecting Malicious Clientscited by 123
2020Radioactive data: tracing through trainingcited by 67
2023FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Modelscited by 63
2024Watermarking Makes Language Models Radioactivecited by 56
2025A Survey on Federated Fine-tuning of Large Language Modelscited by 24

3 more

2022How Much Privacy Does Federated Learning with Secure Aggregation Guarantee?cited by 19
2025Robust Data Watermarking in Language Models by Injecting Fictitious Knowledgecited by 12
2024Tracing Back the Malicious Clients in Poisoning Attacks to Federated Learningcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way