mlsanityData to 14 Oct 2026

Byzantine-Resilient Federated Large Language Model via Hypergraph Message Passing

arXiv 2610.12711 ↗First posted 8 Oct 2026cs.LG cs.CRcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated Large Language Models (FedLLMs) are promising to deliver privacy-preserving, collaborative intelligence. The decentralized training process is vulnerable to Byzantine attacks that corrupt semantic consistency and factual resilience through malicious model updates. This paper presents a new resilience strategy that prevents semantically corrupted knowledge from propagating during federated aggregation. A defense problem is formulated to jointly maximize global accuracy and suppress manipulated model updates, enabling trustworthy knowledge aggregation without accessing private local data. To strengthen the resilience of FedLLMs, we develop a novel Hypergraph Message Passing Graph Autoencoder (HMP-GAE) framework, where low-rank model updates are represented as nodes in a hypergraph and high-order semantic relationships among multiple agents are exploited through iterative message passing. By leveraging historical checkpoints and cross-agent relational reasoning, HMP-GAE identifies and suppresses semantically inconsistent updates while preserving knowledge alignment among agents throughout federated training. HMP-GAE is implemented using Qwen-2.5 and Llama-3.2 as pretrained FedLLM backbones and evaluated on AG News and Yahoo! Answers datasets. Compared to existing defenses, HMP-GAE improves global accuracy by 1.21%, reduces final-model classification semantic entropy by 44.08%, and lowers perplexity by 13.34%. HMP-GAE outperforms the GAE-based defense by improving the accuracy 1.39%, reducing the semantic entropy by 54.5%, and lowering perplexity by 35.5%.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 46 references that are in the map. First posted on arXiv 8 Oct 2026.

Builds on

25 of 46 references.

References · 25

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  3. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  4. 2020FLTrust: Byzantine-robust Federated Learning via Trust Bootstrappingcited by 346
  5. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 223
  6. 2018Mitigating Sybils in Federated Learning Poisoningcited by 161
  7. 2024FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptationscited by 146
  8. 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 99
  9. 2024Advances and Open Challenges in Federated Foundation Modelscited by 36
  10. 2024Federated Large Language Models: Current Progress and Future Directionscited by 27
  11. 2024Federated Fine-Tuning for Pre-Trained Foundation Models Over Wireless Networkscited by 25
  12. 2017Byzantine-Tolerant Machine Learningcited by 19
13 more
  1. 2025LLM-Powered Decentralized Generative Agents with Adaptive Hierarchical Knowledge Graph for Cooperative Planningcited by 18
  2. 2023Hypergraph Neural Networks through the Lens of Message Passing: A Common Perspective to Homophily and Architecture Designcited by 15
  3. 2025FedSRD: Sparsify-Reconstruct-Decompose for Communication-Efficient Federated Large Language Models Fine-Tuningcited by 6
  4. 2025HyperGCT: A Dynamic Hyper-GNN-Learned Geometric Constraint for 3D Registrationcited by 4
  5. 2025Towards Resilient Federated Learning in CyberEdge Networks: Recent Advances and Future Trendscited by 4
  6. 2025A Survey on Foundation Models for Personalized Federated Intelligencecited by 3
  7. 2025Graph Representation-based Model Poisoning on the Heterogeneous Internet of Agentscited by 3
  8. 2025Practical Framework for Privacy-Preserving and Byzantine-robust Federated Learningcited by 3
  9. 2025DSFL: A Dual-Server Byzantine-Resilient Federated Learning Framework via Group-Based Secure Aggregationcited by 2
  10. 2025Perfect Privacy for Discriminator-Based Byzantine-Resilient Federated Learningcited by 1
  11. 2025Efficient Byzantine-Robust Privacy-Preserving Federated Learning via Dimension Compressioncited by 1
  12. 2026Graph Representation Learning Augmented Model Manipulation on Federated Fine-Tuning of LLMscited by 1
  13. 2026Model-Consistent Byzantine-Resilient Decentralized Federated Learning for Collaborative Missionscited by 1

Built on by

Not cited yet.