mlsanityData to 14 Oct 2026

FedRDMA: Communication-Efficient Cross-Silo Federated LLM via Chunked RDMA Transmission

arXiv 2403.00881 ↗First posted 1 Mar 2024cs.LG cs.DC cs.NIcited by 5

Not on the map. Why?

Abstract (arXiv)

Communication overhead is a significant bottleneck in federated learning (FL), which has been exaggerated with the increasing size of AI models. In this paper, we propose FedRDMA, a communication-efficient cross-silo FL system that integrates RDMA into the FL communication protocol. To overcome the limitations of RDMA in wide-area networks (WANs), FedRDMA divides the updated model into chunks and designs a series of optimization techniques to improve the efficiency and robustness of RDMA-based communication. We implement FedRDMA atop the industrial federated learning framework and evaluate it on a real-world cross-silo FL scenario. The experimental results show that sys can achieve up to 3.8× speedup in communication efficiency compared to traditional TCP/IP-based FL systems.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 19 of its 46 references that are in the map. First posted on arXiv 1 Mar 2024.

Builds on

19 of 46 references.

References · 19

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2022Emergent Abilities of Large Language Modelscited by 2,512
  3. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  4. 2020The Future of Digital Health with Federated Learningcited by 633
  5. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 571
  6. 2019Federated Machine Learning: Concept and Applicationscited by 412
  7. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 374
  8. 2023Sentiment Analysis in the Era of Large Language Models: A Reality Checkcited by 268
  9. 2022Leveraging Large Language Models for Multiple Choice Question Answeringcited by 203
  10. 2020FinBERT: A Pretrained Language Model for Financial Communicationscited by 161
  11. 2022From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Modelscited by 115
  12. 2022FedPrompt: Communication-Efficient and Privacy Preserving Prompt Tuning in Federated Learningcited by 91
7 more
  1. 2023Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytescited by 55
  2. 2022Cross-Silo Federated Learning: Challenges and Opportunitiescited by 38
  3. 2023Integration of Large Language Models and Federated Learningcited by 31
  4. 2023Low-Parameter Federated Learning with Large Language Modelscited by 23
  5. 2018Natural Language Processing for EHR-Based Computational Phenotypingcited by 18
  6. 2022Federated Few-Shot Learning for Mobile NLPcited by 15
  7. 2024A Federated Framework for LLM-based Recommendationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 2.

5 papers.

  1. 2024Federated Large Language Models: Current Progress and Future Directionscited by 27
  2. 2026Toward Efficient Membership Inference Attacks against Federated Large Language Models: A Projection Residual Approachcited by 3
  3. 2026ProtegoFed: Backdoor-Free Federated Instruction Tuning with Interspersed Poisoned Datacited by 2
  4. 2025Enhancing Model Privacy in Federated Learning with Random Masking and Quantizationcited by 1
  5. 2025Remote Rowhammer Attack using Adversarial Observations on Federated Learning Clientscited by 0