# Towards Straggler-Resilient Split Federated Learning: An Unbalanced Update Approach

arXiv: https://arxiv.org/abs/2510.21155 · HTML: https://mlsanity.com/paper/2510.21155/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 48 references that are in the map.
- First posted on arXiv 24 Oct 2025.

## Builds on (most cited first, 30 of 34)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming](https://mlsanity.com/paper/1309.5549/) (2013)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [Fine-Tuning Language Models with Just Forward Passes](https://mlsanity.com/paper/2305.17333/) (2023)
- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [SplitFed: When Federated Learning Meets Split Learning](https://mlsanity.com/paper/2004.12088/) (2020)
- [Asynchronous Federated Optimization](https://mlsanity.com/paper/1903.03934/) (2019)
- [Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning](https://mlsanity.com/paper/2101.11203/) (2021)
- [Federated Learning with Buffered Asynchronous Aggregation](https://mlsanity.com/paper/2106.06639/) (2021)
- [A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning](https://mlsanity.com/paper/2006.06224/) (2020)
- [Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregation](https://mlsanity.com/paper/1903.07424/) (2019)
- [Split Learning for collaborative deep learning in healthcare](https://mlsanity.com/paper/1912.12115/) (2019)
- [NoPeek: Information leakage reduction to share activations in distributed deep learning](https://mlsanity.com/paper/2008.09161/) (2020)
- [Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneity](https://mlsanity.com/paper/2012.14453/) (2020)
- [A Zeroth-Order Block Coordinate Descent Algorithm for Huge-Scale Black-Box Optimization](https://mlsanity.com/paper/2102.10707/) (2021)
- [ResSFL: A Resistance Transfer Framework for Defending Model Inversion Attack in Split Federated Learning](https://mlsanity.com/paper/2205.04007/) (2022)
- [FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Update](https://mlsanity.com/paper/2312.05770/) (2023)
- [A Federated Learning Framework for Healthcare IoT devices](https://mlsanity.com/paper/2005.05083/) (2020)
- [Reducing Communication for Split Learning by Randomized Top-k Sparsification](https://mlsanity.com/paper/2305.18469/) (2023)
- [Convergence Analysis of Split Federated Learning on Heterogeneous Data](https://mlsanity.com/paper/2402.15166/) (2024)
- [FLuID: Mitigating Stragglers in Federated Learning using Invariant Dropout](https://mlsanity.com/paper/2307.02623/) (2023)
- [FedLite: A Scalable Approach for Federated Learning on Resource-constrained Clients](https://mlsanity.com/paper/2201.11865/) (2022)
- [FedCompass: Efficient Cross-Silo Federated Learning on Heterogeneous Client Devices using a Computing Power Aware Scheduler](https://mlsanity.com/paper/2309.14675/) (2023)
- [Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization](https://mlsanity.com/paper/2405.15861/) (2024)
- [Communication and Computation Reduction for Split Learning using Asynchronous Training](https://mlsanity.com/paper/2107.09786/) (2021)
- [Have Your Cake and Eat It Too: Toward Efficient and Accurate Split Federated Learning](https://mlsanity.com/paper/2311.13163/) (2023)
- [Convergence Analysis of Sequential Split Learning on Heterogeneous Data](https://mlsanity.com/paper/2302.01633/) (2023)

## Built on by (most cited first, 7 of 7)

- [Quantized-Tinyllava: a new multimodal foundation model enables efficient split learning](https://mlsanity.com/paper/2511.23402/) (2025)
- [HOSL: Hybrid-Order Split Learning for Memory-Constrained Edge Training](https://mlsanity.com/paper/2601.10940/) (2026)
- [Self-Creating Random Walks for Decentralized Learning under Pac-Man Attacks](https://mlsanity.com/paper/2601.07674/) (2026)
- [HO-SFL: Hybrid-Order Split Federated Learning with Backprop-Free Clients and Dimension-Free Aggregation](https://mlsanity.com/paper/2603.14773/) (2026)
- [When Is Shallow Enough? Adaptive Split Federated Learning with Client-Specific Sufficiency Estimation](https://mlsanity.com/paper/2608.15639/) (2026)
- [Latency-Aware Client Assignment for Parallel Split Learning With Global Sampling](https://mlsanity.com/paper/2609.32132/) (2026)
- [CoeF-SFL: Preserving Collaborative Server-Client Learning with Enhanced Communication Efficiency](https://mlsanity.com/paper/2609.34360/) (2026)

## Abstract (arXiv)

Split Federated Learning (SFL) enables scalable training on edge devices by combining the parallelism of Federated Learning (FL) with the computational offloading of Split Learning (SL). Despite its great success, SFL suffers significantly from the well-known straggler issue in distributed learning systems. This problem is exacerbated by the dependency between Split Server and clients: the Split Server side model update relies on receiving activations from clients. Such synchronization requirement introduces significant time latency, making straggler a critical bottleneck to the scalability and efficiency of the system. To mitigate this problem, we propose MU-SplitFed, a straggler-resilient SFL algorithm in zeroth-order optimization that decouples training progress from straggler delays via a simple yet effective unbalanced update mechanism. By enabling the server to perform τ local updates per client round, MU-SplitFed achieves a convergence rate of O(√d/(τ T)) for non-convex objectives, demonstrating a linear speedup of τ in communication rounds. Experiments demonstrate that MU-SplitFed consistently outperforms baseline methods with the presence of stragglers and effectively mitigates their impact through adaptive tuning of τ. The code for this project is available at https://github.com/Johnny-Zip/MU-SplitFed.