# GAPSL: A Gradient-Aligned Parallel Split Learning over Data-Heterogeneous Edge Computing Systems

arXiv: https://arxiv.org/abs/2603.18540 · HTML: https://mlsanity.com/paper/2603.18540/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 49 references that are in the map.
- First posted on arXiv 19 Mar 2026.

## Builds on (most cited first, 30 of 34)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [SplitFed: When Federated Learning Meets Split Learning](https://mlsanity.com/paper/2004.12088/) (2020)
- [Unleashing the Tiger: Inference Attacks on Split Learning](https://mlsanity.com/paper/2012.02670/) (2020)
- [Efficient Parallel Split Learning over Resource-constrained Wireless Edge Networks](https://mlsanity.com/paper/2303.15991/) (2023)
- [Split Learning over Wireless Networks: Parallel Design and Resource Management](https://mlsanity.com/paper/2204.08119/) (2022)
- [Split Learning in 6G Edge Networks](https://mlsanity.com/paper/2306.12194/) (2023)
- [AdaptSFL: Adaptive Split Federated Learning in Resource-constrained Edge Networks](https://mlsanity.com/paper/2403.13101/) (2024)
- [Automated Federated Pipeline for Parameter-Efficient Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2404.06448/) (2024)
- [FedSN: A Federated Learning Framework over Heterogeneous LEO Satellite Networks](https://mlsanity.com/paper/2311.01483/) (2023)
- [Hierarchical Split Federated Learning: Convergence Analysis and System Optimization](https://mlsanity.com/paper/2412.07197/) (2024)
- [Towards Personalized Federated Learning via Heterogeneous Model Reassembly](https://mlsanity.com/paper/2308.08643/) (2023)
- [HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models](https://mlsanity.com/paper/2505.02795/) (2025)
- [Server-Side Local Gradient Averaging and Learning Rate Acceleration for Scalable Split Learning](https://mlsanity.com/paper/2112.05929/) (2021)
- [Unleashing the Power of Continual Learning on Non-Centralized Devices: A Survey](https://mlsanity.com/paper/2412.13840/) (2024)
- [A Bargaining Game for Personalized, Energy Efficient Split Learning over Wireless Networks](https://mlsanity.com/paper/2212.06107/) (2022)
- [LEO-Split: A Semi-Supervised Split Learning Framework over LEO Satellite Networks](https://mlsanity.com/paper/2501.01293/) (2025)
- [FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors](https://mlsanity.com/paper/2503.15842/) (2025)
- [FedCache: A Knowledge Cache-driven Federated Learning Architecture for Personalized Edge Intelligence](https://mlsanity.com/paper/2308.07816/) (2023)
- [HASFL: Heterogeneity-aware Split Federated Learning over Edge Computing Systems](https://mlsanity.com/paper/2506.08426/) (2025)
- [Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Data](https://mlsanity.com/paper/2405.06413/) (2024)
- [FedCiR: Client-Invariant Representation Learning for Federated Non-IID Features](https://mlsanity.com/paper/2308.15786/) (2023)
- [Splitfed learning without client-side synchronization: Analyzing client-side split network portion size to overall performance](https://mlsanity.com/paper/2109.09246/) (2021)
- [Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring](https://mlsanity.com/paper/2508.09085/) (2025)
- [Actions at the Edge: Jointly Optimizing the Resources in Multi-access Edge Computing](https://mlsanity.com/paper/2204.08169/) (2022)
- [FSL-SAGE: Accelerating Federated Split Learning via Smashed Activation Gradient Estimation](https://mlsanity.com/paper/2505.23182/) (2025)
- [Faster Convergence on Heterogeneous Federated Edge Learning: An Adaptive Clustered Data Sharing Approach](https://mlsanity.com/paper/2406.09776/) (2024)
- [Parallel Split Learning with Global Sampling](https://mlsanity.com/paper/2407.15738/) (2024)

## Abstract (arXiv)

The increasing complexity of neural networks poses significant challenges for democratizing federated learning (FL) on resource-constrained edge devices. Parallel split learning (PSL) has emerged as a promising solution by offloading substantial computing workload to a server via model partitioning, shrinking client-side computing load, and eliminating the client-side model aggregation for reduced communication and deployment costs. However, the highly heterogeneous nature of client data in edge computing systems causes aggregation-free PSL to suffer from severe training divergence, stemming from gradient directional inconsistency across clients. To address this challenge, we propose GAPSL, a gradient-aligned PSL framework tailored for data-heterogeneous edge systems, which comprises two key components: leader gradient identification (LGI) and gradient direction alignment (GDA). LGI dynamically selects a set of directionally consistent device gradients to construct a leader gradient as a robust proxy for the global convergence trend. GDA employs a direction-aware regularization to align each client's gradient with the leader gradient, thereby mitigating inter-device gradient directional inconsistency and enhancing model convergence. We evaluate GAPSL on a prototype computing testbed. Extensive experiments demonstrate that GAPSL consistently outperforms state-of-the-art benchmarks in training accuracy, convergence latency, and system robustness under severe data heterogeneity.