# Byzantine-Resilient Federated Large Language Model via Hypergraph Message Passing

arXiv: https://arxiv.org/abs/2610.12711 · HTML: https://mlsanity.com/paper/2610.12711/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 8 Oct 2026.

## Builds on (most cited first, 25 of 25)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping](https://mlsanity.com/paper/2012.13995/) (2020)
- [G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering](https://mlsanity.com/paper/2402.07630/) (2024)
- [Mitigating Sybils in Federated Learning Poisoning](https://mlsanity.com/paper/1808.04866/) (2018)
- [FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations](https://mlsanity.com/paper/2409.05976/) (2024)
- [Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark](https://mlsanity.com/paper/2311.06750/) (2023)
- [Advances and Open Challenges in Federated Foundation Models](https://mlsanity.com/paper/2404.15381/) (2024)
- [Federated Large Language Models: Current Progress and Future Directions](https://mlsanity.com/paper/2409.15723/) (2024)
- [Federated Fine-Tuning for Pre-Trained Foundation Models Over Wireless Networks](https://mlsanity.com/paper/2407.02924/) (2024)
- [Byzantine-Tolerant Machine Learning](https://mlsanity.com/paper/1703.02757/) (2017)
- [LLM-Powered Decentralized Generative Agents with Adaptive Hierarchical Knowledge Graph for Cooperative Planning](https://mlsanity.com/paper/2502.05453/) (2025)
- [Hypergraph Neural Networks through the Lens of Message Passing: A Common Perspective to Homophily and Architecture Design](https://mlsanity.com/paper/2310.07684/) (2023)
- [FedSRD: Sparsify-Reconstruct-Decompose for Communication-Efficient Federated Large Language Models Fine-Tuning](https://mlsanity.com/paper/2510.04601/) (2025)
- [HyperGCT: A Dynamic Hyper-GNN-Learned Geometric Constraint for 3D Registration](https://mlsanity.com/paper/2503.02195/) (2025)
- [Towards Resilient Federated Learning in CyberEdge Networks: Recent Advances and Future Trends](https://mlsanity.com/paper/2504.01240/) (2025)
- [A Survey on Foundation Models for Personalized Federated Intelligence](https://mlsanity.com/paper/2505.06907/) (2025)
- [Graph Representation-based Model Poisoning on the Heterogeneous Internet of Agents](https://mlsanity.com/paper/2511.07176/) (2025)
- [Practical Framework for Privacy-Preserving and Byzantine-robust Federated Learning](https://mlsanity.com/paper/2512.17254/) (2025)
- [DSFL: A Dual-Server Byzantine-Resilient Federated Learning Framework via Group-Based Secure Aggregation](https://mlsanity.com/paper/2509.08449/) (2025)
- [Perfect Privacy for Discriminator-Based Byzantine-Resilient Federated Learning](https://mlsanity.com/paper/2506.13561/) (2025)
- [Efficient Byzantine-Robust Privacy-Preserving Federated Learning via Dimension Compression](https://mlsanity.com/paper/2509.11870/) (2025)
- [Graph Representation Learning Augmented Model Manipulation on Federated Fine-Tuning of LLMs](https://mlsanity.com/paper/2605.07961/) (2026)
- [Model-Consistent Byzantine-Resilient Decentralized Federated Learning for Collaborative Missions](https://mlsanity.com/paper/2608.22552/) (2026)

## Abstract (arXiv)

Federated Large Language Models (FedLLMs) are promising to deliver privacy-preserving, collaborative intelligence. The decentralized training process is vulnerable to Byzantine attacks that corrupt semantic consistency and factual resilience through malicious model updates. This paper presents a new resilience strategy that prevents semantically corrupted knowledge from propagating during federated aggregation. A defense problem is formulated to jointly maximize global accuracy and suppress manipulated model updates, enabling trustworthy knowledge aggregation without accessing private local data. To strengthen the resilience of FedLLMs, we develop a novel Hypergraph Message Passing Graph Autoencoder (HMP-GAE) framework, where low-rank model updates are represented as nodes in a hypergraph and high-order semantic relationships among multiple agents are exploited through iterative message passing. By leveraging historical checkpoints and cross-agent relational reasoning, HMP-GAE identifies and suppresses semantically inconsistent updates while preserving knowledge alignment among agents throughout federated training. HMP-GAE is implemented using Qwen-2.5 and Llama-3.2 as pretrained FedLLM backbones and evaluated on AG News and Yahoo! Answers datasets. Compared to existing defenses, HMP-GAE improves global accuracy by 1.21%, reduces final-model classification semantic entropy by 44.08%, and lowers perplexity by 13.34%. HMP-GAE outperforms the GAE-based defense by improving the accuracy 1.39%, reducing the semantic entropy by 54.5%, and lowering perplexity by 35.5%.