# Large Language Model Federated Learning with Blockchain and Unlearning for Cross-Organizational Collaboration

arXiv: https://arxiv.org/abs/2412.13551 · HTML: https://mlsanity.com/paper/2412.13551/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security.
- Builds on 16 of its 39 references that are in the map.
- First posted on arXiv 18 Dec 2024.

## Placed in

- LLM safety and security / [Federated unlearning and poisoning](https://mlsanity.com/safety/federated-unlearning-and-poisoning/)

## Builds on (most cited first, 16 of 16)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [TOFU: A Task of Fictitious Unlearning for LLMs](https://mlsanity.com/paper/2401.06121/) (2024)
- [Who's Harry Potter? Approximate Unlearning in LLMs](https://mlsanity.com/paper/2310.02238/) (2023)
- [Large Language Model Unlearning](https://mlsanity.com/paper/2310.10683/) (2023)
- [Unlearn What You Want to Forget: Efficient Unlearning for LLMs](https://mlsanity.com/paper/2310.20150/) (2023)
- [FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning](https://mlsanity.com/paper/2309.00363/) (2023)
- [Towards Safer Large Language Models through Machine Unlearning](https://mlsanity.com/paper/2402.10058/) (2024)
- [OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning](https://mlsanity.com/paper/2402.06954/) (2024)
- [FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Model](https://mlsanity.com/paper/2406.17706/) (2024)
- [FederatedTrust: A Solution for Trustworthy Federated Learning](https://mlsanity.com/paper/2302.09844/) (2023)
- [BC4LLM: Trusted Artificial Intelligence When Blockchain Meets Large Language Models](https://mlsanity.com/paper/2310.06278/) (2023)
- [Blockchain-enabled Trustworthy Federated Unlearning](https://mlsanity.com/paper/2401.15917/) (2024)
- [A Federated Framework for LLM-based Recommendation](https://mlsanity.com/paper/2402.09959/) (2024)
- [Blockchain Mutability: Challenges and Proposed Solutions](https://mlsanity.com/paper/1907.07099/) (2019)
- [Dynamic Large Language Models on Blockchains](https://mlsanity.com/paper/2307.10549/) (2023)
- [A Comprehensive Survey on Edge Data Integrity Verification: Fundamentals and Future Trends](https://mlsanity.com/paper/2210.10978/) (2022)

## Built on by (most cited first, 1 of 1)

- [SoK: Privacy-aware LLM in Healthcare: Threat Model, Privacy Techniques, Challenges and Recommendations](https://mlsanity.com/paper/2601.10004/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have transformed the way computers understand and process human language, but using them effectively across different organizations remains still difficult. When organizations work together to improve LLMs, they face several main challenges. First, organizations hesitate to share their valuable data with others. Second, competition between organizations creates trust problems during collaboration. Third, new privacy laws require organizations to be able to delete specific data when requested, which is especially difficult when multiple organizations are learning from shared data. Traditional federated learning approaches do not address these interconnected challenges, particularly in scenarios where participants cannot fully trust each other or the central aggregator. To overcome these limitations, we propose a hybrid blockchain-based federated learning framework that uniquely combines public and private blockchain architectures with multi-agent reinforcement learning. Our framework enables transparent sharing of model update through the public blockchain while protecting sensitive computations in private chains. Each organization operates as an intelligent agent, using Q-learning to optimize its participation strategy and resource allocation, thus aligning individual incentives with collective goals. Notably, we introduce an efficient unlearning mechanism based on Low-Rank Adaptation (LoRA) that enables selective removal of specific data contributions without compromising the model's overall performance. Through extensive experimentation on real-world datasets, we demonstrate that our framework effectively balances privacy protection, trust establishment, and regulatory compliance while maintaining high model performance.