mlsanityData to 7 Oct 2026

Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly

arXiv 2310.03150 ↗First posted 4 Oct 2023cs.LG cs.DC cs.PFcited by 17

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLM) and foundation models are popular as they offer new opportunities for individuals and businesses to improve natural language processing, interact with data, and retrieve information faster. However, training or fine-tuning LLMs requires a vast amount of data, which can be challenging to access due to legal or technical restrictions and may require private computing resources. Federated Learning (FL) is a solution designed to overcome these challenges and expand data access for deep learning applications. This paper takes a hardware-centric approach to explore how LLMs can be brought to modern edge computing systems. Our study fine-tunes the FLAN-T5 model family, ranging from 80M to 3B parameters, using FL for a text summarization task. We provide a micro-level hardware benchmark, compare the model FLOP utilization to a state-of-the-art data center GPU, and study the network utilization in realistic conditions. Our contribution is twofold: First, we evaluate the current capabilities of edge computing systems and their potential for LLM FL workloads. Second, by comparing these systems with a data-center GPU, we demonstrate the potential for improvement and the next steps toward achieving greater computational efficiency at the edge.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 27 of its 54 references that are in the map. First posted on arXiv 4 Oct 2023.

Builds on

27 of 54 references.

References · 27

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  4. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  5. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  6. 2020Adaptive Federated Optimizationcited by 1,036
  7. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  8. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  9. 2018LEAF: A Benchmark for Federated Settingscited by 780
  10. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
  11. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 530
  12. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
15 more
  1. 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 339
  2. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
  3. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 184
  4. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 135
  5. 2020Data Movement Is All You Need: A Case Study on Optimizing Transformerscited by 108
  6. 2023SLoRA: Federated Parameter Efficient Fine-Tuning of Language Modelscited by 92
  7. 2018In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learningcited by 80
  8. 2019Accelerating Federated Learning via Momentum Gradient Descentcited by 75
  9. 2022When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methodscited by 74
  10. 2021Compute and Energy Consumption Trends in Deep Learning Inferencecited by 61
  11. 2023FwdLLM: Efficient FedLLM using Forward Gradientcited by 25
  12. 2022Accelerated Federated Learning with Decoupled Adaptive Optimizationcited by 23
  13. 2023Federated Learning with Regularized Client Participationcited by 13
  14. 2022Benchmarking Resource Usage for Efficient Distributed Deep Learningcited by 7
  15. 2021ERRANT: Realistic Emulation of Radio Access Networkscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 8; 2025: 6; 2026: 1.

17 papers.

Sort:

By area:

  1. 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 125
  2. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  3. 2023Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytescited by 55
  4. 2024A Survey on Efficient Federated Learning Methods for Foundation Model Trainingcited by 29
  5. 2023Green Edge AI: A Contemporary Surveycited by 24
  6. 2024Federated Large Language Models: Current Progress and Future Directionscited by 22
  7. 2024Training Machine Learning models at the Edge: A Surveycited by 11
  8. 2025Adaptive Rank Allocation for Federated Parameter-Efficient Fine-Tuning of Language Modelscited by 4
  9. 2025Towards Harnessing the Collaborative Power of Large and Small Models for Domain Taskscited by 4
  10. 2024Snake Learning: A Communication- and Computation-Efficient Distributed Learning Framework for 6Gcited by 3
  11. 2024Revisiting Federated Fine-Tuning: A Single Communication Round is Enough for Foundation Modelscited by 1
  12. 2025FedRPCA: Enhancing Federated LoRA Aggregation Using Robust PCAcited by 1
5 more
  1. 2024TaskComplexity: A Dataset for Task Complexity Classification with In-Context Learning, FLAN-T5 and GPT-4o Benchmarkscited by 0
  2. 2025FDA-Opt: Federated Fine-Tuning via Dynamic Update Schedulescited by 0
  3. 2025ATR-Bench: A Federated Learning Benchmark for Adaptation, Trust, and Reasoningcited by 0
  4. 2025FLAME: Towards Federated Fine-Tuning Large Language Models Through Adaptive SMoEcited by 0
  5. 2026Energy-Efficient GPU DVFS for Fine-Tuning of SLMs on Resource-constrained Embedded Devicescited by 0