# Federated TrustChain: Blockchain-Enhanced LLM Training and Unlearning

arXiv: https://arxiv.org/abs/2406.04076 · HTML: https://mlsanity.com/paper/2406.04076/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security.
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 6 Jun 2024.

## Placed in

- LLM safety and security / [Federated unlearning and poisoning](https://mlsanity.com/safety/federated-unlearning-and-poisoning/)

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)
- [Large Language Model Unlearning](https://mlsanity.com/paper/2310.10683/) (2023)
- [FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning](https://mlsanity.com/paper/2309.00363/) (2023)
- [KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment](https://mlsanity.com/paper/2305.06535/) (2023)
- [Recovering Private Text in Federated Learning of Language Models](https://mlsanity.com/paper/2205.08514/) (2022)
- [FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Models](https://mlsanity.com/paper/2310.10049/) (2023)
- [Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges](https://mlsanity.com/paper/2311.15766/) (2023)
- [Offset Unlearning for Large Language Models](https://mlsanity.com/paper/2404.11045/) (2024)
- [Integration of Large Language Models and Federated Learning](https://mlsanity.com/paper/2307.08925/) (2023)
- [Low-Parameter Federated Learning with Large Language Models](https://mlsanity.com/paper/2307.13896/) (2023)
- [Personalized Wireless Federated Learning for Large Language Models](https://mlsanity.com/paper/2404.13238/) (2024)
- [Enhancing Trust and Privacy in Distributed Networks: A Comprehensive Survey on Blockchain-based Federated Learning](https://mlsanity.com/paper/2403.19178/) (2024)
- [BC4LLM: Trusted Artificial Intelligence When Blockchain Meets Large Language Models](https://mlsanity.com/paper/2310.06278/) (2023)
- [LLMChain: Blockchain-based Reputation System for Sharing and Evaluating Large Language Models](https://mlsanity.com/paper/2404.13236/) (2024)
- [Class Machine Unlearning for Complex Data via Concepts Inference and Data Poisoning](https://mlsanity.com/paper/2405.15662/) (2024)
- [Efficient Language Model Architectures for Differentially Private Federated Learning](https://mlsanity.com/paper/2403.08100/) (2024)
- [Generative AI-enabled Blockchain Networks: Fundamentals, Applications, and Case Study](https://mlsanity.com/paper/2401.15625/) (2024)

## Built on by (most cited first, 3 of 3)

- [Blockchain for Large Language Model Security and Safety: A Holistic Survey](https://mlsanity.com/paper/2407.20181/) (2024)
- [Reliable Unlearning Harmful Information in LLMs with Metamorphosis Representation Projection](https://mlsanity.com/paper/2508.15449/) (2025)
- [A Scenario-Oriented Survey of Federated Recommender Systems: Techniques, Challenges, and Future Directions](https://mlsanity.com/paper/2508.19620/) (2025)

## Abstract (arXiv)

The development of Large Language Models (LLMs) faces a significant challenge: the exhausting of publicly available fresh data. This is because training a LLM needs a large demanding of new data. Federated learning emerges as a promising solution, enabling collaborative model to contribute their private data to LLM global model. However, integrating federated learning with LLMs introduces new challenges, including the lack of transparency and the need for effective unlearning mechanisms. Transparency is essential to ensuring trust and fairness among participants, while accountability is crucial for deterring malicious behaviour and enabling corrective actions when necessary. To address these challenges, we propose a novel blockchain-based federated learning framework for LLMs that enhances transparency, accountability, and unlearning capabilities. Our framework leverages blockchain technology to create a tamper-proof record of each model's contributions and introduces an innovative unlearning function that seamlessly integrates with the federated learning mechanism. We investigate the impact of Low-Rank Adaptation (LoRA) hyperparameters on unlearning performance and integrate Hyperledger Fabric to ensure the security, transparency, and verifiability of the unlearning process. Through comprehensive experiments and analysis, we showcase the effectiveness of our proposed framework in achieving highly effective unlearning in LLMs trained using federated learning. Our findings highlight the feasibility of integrating blockchain technology into federated learning frameworks for LLMs.