mlsanityData to 7 Oct 2026

eFedLLM: Efficient LLM Inference Based on Federated Learning

arXiv 2411.16003 ↗First posted 24 Nov 2024cs.LG cs.AIcited by 3

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Large Language Models (LLMs) herald a transformative era in artificial intelligence (AI). However, the expansive scale of data and parameters of LLMs requires high-demand computational and memory resources, restricting their accessibility to a broader range of users and researchers. This paper introduces an effective approach that enhances the operational efficiency and affordability of LLM inference. By utilizing transformer-based federated learning (FL) with model-parallel distributed training, our model efficiently distributes the computational loads and memory requirements across a network of participants. This strategy permits users, especially those with limited resources to train state-of-the-art LLMs collaboratively. We also innovate an incentive mechanism within the FL framework, rewarding constructive contributions and filtering out malicious activities, thereby safeguarding the integrity and reliability of the training process. Concurrently, we leverage memory hierarchy strategies and Singular Value Decomposition (SVD) on weight matrices to boost computational and memory efficiencies further. Our results, derived from formulaic analyses and numerical calculations, demonstrate significant optimization of resource use and democratize access to cutting-edge LLMs, ensuring that a wide scale of users can both contribute to and benefit from these advanced models.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 22 of its 29 references that are in the map. First posted on arXiv 24 Nov 2024.

Builds on

22 of 29 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  8. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  9. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  10. 2019Local Model Poisoning Attacks to Byzantine-Robust Federated Learningcited by 479
  11. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
  12. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 371
10 more
  1. 2018A Hybrid Approach to Privacy-Preserving Federated Learningcited by 208
  2. 2021Generating Datasets with Pretrained Language Modelscited by 192
  3. 2022Petals: Collaborative Inference and Fine-tuning of Large Modelscited by 84
  4. 2024opML: Optimistic Machine Learning on Blockchaincited by 20
  5. 2021Practical Lossless Federated Singular Vector Decomposition over Billion-Scale Datacited by 18
  6. 2024Exploring Vulnerabilities and Protections in Large Language Models: A Surveycited by 10
  7. 2017A Zero Knowledge Sumcheck and its Applicationscited by 4
  8. 2022Survey on the Convergence of Machine Learning and Blockchaincited by 4
  9. 2022An Investigation of Smart Contract for Collaborative Machine Learning Model Trainingcited by 1
  10. 2024CFSafety: Comprehensive Fine-grained Safety Assessment for LLMscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025MobileA3gent: Training Mobile GUI Agents Using Decentralized Self-Sourced Data from Diverse Userscited by 10
  2. 2026SoK: Privacy-aware LLM in Healthcare: Threat Model, Privacy Techniques, Challenges and Recommendationscited by 1
  3. 2026ELSA: Efficient LLM-Centric Split Aggregation for Privacy-Aware Hierarchical Federated Learning over the Network Edgecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Taming the Titans: A Survey of Efficient LLM Inference Serving
  2. 2025An Explorative Study on Distributed Computing Techniques in Training and Inference of Large Language Models ↗
  3. 2024LLM Inference Unveiled: Survey and Roofline Model Insights
  4. 2025Speed Always Wins: A Survey on Efficient Architectures for Large Language Models
  5. 2024A Survey on Efficient Inference for Large Language Models
  6. 2025Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques
  7. 2025Towards Efficient Multi-LLM Inference: Characterization and Analysis of LLM Routing and Hierarchical Techniques
  8. 2023Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline