mlsanityData to 14 Oct 2026

Server-Side Local Gradient Averaging and Learning Rate Acceleration for Scalable Split Learning

arXiv 2112.05929 ↗First posted 11 Dec 2021cs.LG cs.AIcited by 23

Not on the map. Why?

Abstract (arXiv)

In recent years, there have been great advances in the field of decentralized learning with private data. Federated learning (FL) and split learning (SL) are two spearheads possessing their pros and cons, and are suited for many user clients and large models, respectively. To enjoy both benefits, hybrid approaches such as SplitFed have emerged of late, yet their fundamentals have still been illusive. In this work, we first identify the fundamental bottlenecks of SL, and thereby propose a scalable SL framework, coined SGLR. The server under SGLR broadcasts a common gradient averaged at the split-layer, emulating FL without any additional communication across clients as opposed to SplitFed. Meanwhile, SGLR splits the learning rate into its server-side and client-side rates, and separately adjusts them to support many clients in parallel. Simulation results corroborate that SGLR achieves higher accuracy than other baseline SL methods including SplitFed, which is even on par with FL consuming higher energy and communication costs. As a secondary result, we observe greater reduction in leakage of sensitive information via mutual information using SLGR over the baselines.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 23 of its 25 references that are in the map. First posted on arXiv 11 Dec 2021.

Builds on

23 of 25 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  7. 2016Densely Connected Convolutional Networkscited by 6,720
  8. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  9. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  10. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
  11. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
  12. 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 395
11 more
  1. 2017Don't Decay the Learning Rate, Increase the Batch Sizecited by 320
  2. 2020SplitFed: When Federated Learning Meets Split Learningcited by 310
  3. 2018Distributed learning of deep neural network over multiple agentscited by 279
  4. 2014One weird trick for parallelizing convolutional neural networkscited by 264
  5. 2018Wireless Network Intelligence at the Edgecited by 92
  6. 2021Larger-Scale Transformers for Multilingual Masked Language Modelingcited by 87
  7. 2018The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approachescited by 75
  8. 2021Revisiting Locally Supervised Learning: an Alternative to End-to-end Trainingcited by 71
  9. 2020Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applicationscited by 62
  10. 2020A Framework For Contrastive Self-Supervised Learning And Designing A New Approachcited by 52
  11. 2021Faithful Edge Federated Learning: Scalability and Privacycited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 4; 2024: 6; 2025: 6; 2026: 3.

23 papers.

Sort:

By area:

  1. 2023Efficient Parallel Split Learning over Resource-constrained Wireless Edge Networkscited by 95
  2. 2024AdaptSFL: Adaptive Split Federated Learning in Resource-constrained Edge Networkscited by 64
  3. 2025HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Modelscited by 31
  4. 2022Combined Federated and Split Learning in Edge Computing for Ubiquitous Intelligence in Internet of Things: State of the Art and Future Directionscited by 14
  5. 2023MergeSFL: Split Federated Learning with Feature Merging and Batch Size Regulationcited by 13
  6. 2022Visual Transformer Meets CutMix for Improved Accuracy, Communication Efficiency, and Data Privacy in Split Learningcited by 8
  7. 2024Federated Split Learning with Model Pruning and Gradient Quantization in Wireless Networkscited by 7
  8. 2022Local Learning on Transformers via Feature Reconstructioncited by 4
  9. 2024Breaking the Memory Wall for Heterogeneous Federated Learning via Model Splittingcited by 4
  10. 2023PFSL: Personalized & Fair Split Learning with Data & Label Privacy for thin clientscited by 3
  11. 2023SLPerf: a Unified Framework for Benchmarking Split Learningcited by 3
  12. 2024Parallel Split Learning with Global Samplingcited by 3
11 more
  1. 2022PiPar: Pipeline Parallelism for Collaborative Machine Learningcited by 2
  2. 2024Privacy-Preserving Split Learning with Vision Transformers using Patch-Wise Random and Noisy CutMixcited by 2
  3. 2025P3SL: Personalized Privacy-Preserving Split Learning on Heterogeneous Edge Devicescited by 2
  4. 2025Fine-tuning Multimodal Transformers on Edge: A Parallel Split Learning Approachcited by 1
  5. 2025CycleSL: Server-Client Cyclical Update Driven Scalable Split Learningcited by 1
  6. 2024Non-Federated Multi-Task Split Learning for Heterogeneous Sourcescited by 0
  7. 2025Traversal Learning: A Lossless And Efficient Distributed Learning Frameworkcited by 0
  8. 2025Enhancing Split Learning with Sharded and Blockchain-Enabled SplitFed Approachescited by 0
  9. 2026GAPSL: A Gradient-Aligned Parallel Split Learning over Data-Heterogeneous Edge Computing Systemscited by 0
  10. 2026A Survey on Split Learning for LLM Fine-Tuning: Models, Systems, and Privacy Optimizationscited by 0
  11. 2026Latency-Aware Client Assignment for Parallel Split Learning With Global Samplingcited by 0