mlsanityData to 7 Oct 2026

FedLite: A Scalable Approach for Federated Learning on Resource-constrained Clients

arXiv 2201.11865 ↗First posted 28 Jan 2022cs.LG cs.DCcited by 14

Not on the map. Why?

Abstract (arXiv)

In classical federated learning, the clients contribute to the overall training by communicating local updates for the underlying model on their private data to a coordinating server. However, updating and communicating the entire model becomes prohibitively expensive when resource-constrained clients collectively aim to train a large machine learning model. Split learning provides a natural solution in such a setting, where only a small part of the model is stored and trained on clients while the remaining large part of the model only stays at the servers. However, the model partitioning employed in split learning introduces a significant amount of communication cost. This paper addresses this issue by compressing the additional communication using a novel clustering scheme accompanied by a gradient correction method. Extensive empirical evaluations on image and text benchmarks show that the proposed method can achieve up to 490× communication cost reduction with minimal drop in accuracy, and enables a desirable performance vs. communication trade-off.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 24 of its 44 references that are in the map. First posted on arXiv 28 Jan 2022.

Builds on

24 of 44 references.

References · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  5. 2020Adaptive Federated Optimizationcited by 1,036
  6. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  7. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  8. 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 391
  9. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  10. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  11. 2021A Field Guide to Federated Optimizationcited by 312
  12. 2020SplitFed: When Federated Learning Meets Split Learningcited by 308
12 more
  1. 2014One weird trick for parallelizing convolutional neural networkscited by 264
  2. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 234
  3. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  4. 2016QSGD: Communication-Efficient SGD via Gradient Quantization and Encodingcited by 147
  5. 2020Minibatch vs Local SGD for Heterogeneous Distributed Learningcited by 147
  6. 2018ATOMO: Communication-efficient Learning via Atomic Sparsificationcited by 135
  7. 2016Distributed Mean Estimation with Limited Communicationcited by 134
  8. 2020InstaHide: Instance-hiding Schemes for Private Distributed Learningcited by 90
  9. 2019Differentiable Product Quantization for End-to-End Embedding Compressioncited by 36
  10. 2021PyVertical: A Vertical Federated Learning Framework for Multi-headed SplitNNcited by 35
  11. 2021FedJAX: Federated learning simulation with JAXcited by 32
  12. 2021On the Utility of Gradient Compression in Distributed Training Systemscited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 5; 2024: 3; 2025: 1; 2026: 2.

14 papers.

Sort:

  1. 2023Communication-Efficient Split Learning via Adaptive Feature-Wise Compressioncited by 18
  2. 2023FLuID: Mitigating Stragglers in Federated Learning using Invariant Dropoutcited by 16
  3. 2022Combined Federated and Split Learning in Edge Computing for Ubiquitous Intelligence in Internet of Things: State of the Art and Future Directionscited by 14
  4. 2023Federated Learning While Providing Model as a Service: Joint Training and Inference Optimizationcited by 8
  5. 2025Towards Straggler-Resilient Split Federated Learning: An Unbalanced Update Approachcited by 7
  6. 2022Reconciling Security and Communication Efficiency in Federated Learningcited by 4
  7. 2024SplitFedZip: Learned Compression for Data Transfer Reduction in Split-Federated Learningcited by 4
  8. 2024GAS: Generative Activation-Aided Asynchronous Split Federated Learningcited by 3
  9. 2024SCALA: Split Federated Learning with Concatenated Activations and Logit Adjustmentscited by 2
  10. 2023Collaborative Distributed Machine Learningcited by 1
  11. 2023A Novel Federated Learning-Based IDS for Enhancing UAVs Privacy and Securitycited by 1
  12. 2022Reducing Impacts of System Heterogeneity in Federated Learning using Weight Update Magnitudescited by 0
2 more
  1. 2026SplitLite: Low-Rank Residual Compression for Split Learningcited by 0
  2. 2026Importance-Aware Feature Sparsification for Wireless Split Learningcited by 0