# A Survey on Split Learning for LLM Fine-Tuning: Models, Systems, and Privacy Optimizations

arXiv: https://arxiv.org/abs/2604.24468 · HTML: https://mlsanity.com/paper/2604.24468/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 183 references that are in the map.
- First posted on arXiv 27 Apr 2026.

## Builds on (most cited first, 29 of 29)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [iDLG: Improved Deep Leakage from Gradients](https://mlsanity.com/paper/2001.02610/) (2020)
- [SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Models](https://mlsanity.com/paper/2407.00952/) (2024)
- [HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models](https://mlsanity.com/paper/2505.02795/) (2025)
- [Differentially Private Label Protection in Split Learning](https://mlsanity.com/paper/2203.02073/) (2022)
- [Server-Side Local Gradient Averaging and Learning Rate Acceleration for Scalable Split Learning](https://mlsanity.com/paper/2112.05929/) (2021)
- [Feature Space Hijacking Attacks against Differentially Private Split Learning](https://mlsanity.com/paper/2201.04018/) (2022)
- [FedLite: A Scalable Approach for Federated Learning on Resource-constrained Clients](https://mlsanity.com/paper/2201.11865/) (2022)
- [Splitfed learning without client-side synchronization: Analyzing client-side split network portion size to overall performance](https://mlsanity.com/paper/2109.09246/) (2021)
- [Split HE: Fast Secure Inference Combining Split Learning and Homomorphic Encryption](https://mlsanity.com/paper/2202.13351/) (2022)
- [Label Inference Attack against Split Learning under Regression Setting](https://mlsanity.com/paper/2301.07284/) (2023)
- [Have Your Cake and Eat It Too: Toward Efficient and Accurate Split Federated Learning](https://mlsanity.com/paper/2311.13163/) (2023)
- [CURE: Privacy-Preserving Split Learning Done Right](https://mlsanity.com/paper/2407.08977/) (2024)
- [SFPrompt: Communication-Efficient Split Federated Fine-Tuning for Large Pre-Trained Models over Resource-Limited Devices](https://mlsanity.com/paper/2407.17533/) (2024)
- [Mutual Information Regularization for Vertical Federated Learning](https://mlsanity.com/paper/2301.01142/) (2023)
- [When MiniBatch SGD Meets SplitFed Learning:Convergence Analysis and Performance Evaluation](https://mlsanity.com/paper/2308.11953/) (2023)
- [Evaluating Privacy Leakage in Split Learning](https://mlsanity.com/paper/2305.12997/) (2023)
- [SCALA: Split Federated Learning with Concatenated Activations and Logit Adjustments](https://mlsanity.com/paper/2405.04875/) (2024)
- [Dullahan: Stealthy Backdoor Attack against Without-Label-Sharing Split Learning](https://mlsanity.com/paper/2405.12751/) (2024)
- [SFL-LEO: Asynchronous Split-Federated Learning Design for LEO Satellite-Ground Network Framework](https://mlsanity.com/paper/2504.13479/) (2025)
- [A Taxonomy of Attacks and Defenses in Split Learning](https://mlsanity.com/paper/2505.05872/) (2025)
- [Ampere: Communication-Efficient and High-Accuracy Split Federated Learning](https://mlsanity.com/paper/2507.07130/) (2025)
- [FlexP-SFT: A Flexible Aggregation-Free Framework for On-Device Personalized Split Federated Fine-Tuning of LLMs](https://mlsanity.com/paper/2508.10349/) (2025)
- [Differentially Private and Communication Efficient Large Language Model Split Inference via Stochastic Quantization and Soft Prompt](https://mlsanity.com/paper/2602.11513/) (2026)
- [An Efficient Split Fine-tuning Framework for Edge and Cloud Collaborative Learning](https://mlsanity.com/paper/2211.16703/) (2022)
- [Communication-Computation Pipeline Parallel Split Learning over Wireless Edge Networks](https://mlsanity.com/paper/2511.23167/) (2025)

## Abstract (arXiv)

Fine-tuning unlocks large language models (LLMs) for specialized applications, but its high computational cost often puts it out of reach for resource-constrained organizations. While cloud platforms could provide the needed resources, data privacy concerns make sharing sensitive information with third parties risky. A promising solution is split learning for LLM fine-tuning, which divides the model between clients and a server, allowing collaborative and secure training through exchanged intermediate data, thus enabling resource-constrained participants to adapt LLMs safely. % In light of this, a growing body of literature has emerged to advance this paradigm, introducing varied model methods, system optimizations, and privacy defense-attack techniques for split learning. To bring clarity and direction to the field, a comprehensive survey is needed to classify, compare, and critique these diverse approaches. This paper fills the gap by presenting the first extensive survey dedicated to split learning for LLM fine-tuning. We propose a unified, fine-grained training pipeline to pinpoint key operational components and conduct a systematic review of state-of-the-art work across three core dimensions: model-level optimization, system-level efficiency, and privacy preservation. Through this structured taxonomy, we establish a foundation for advancing scalable, robust, and secure collaborative LLM adaptation.