mlsanityData to 13 Oct 2026

Reducing Communication for Split Learning by Randomized Top-k Sparsification

arXiv 2305.18469 ↗First posted 29 May 2023cs.LG cs.DCcited by 22

Not on the map. Why?

Abstract (arXiv)

Split learning is a simple solution for Vertical Federated Learning (VFL), which has drawn substantial attention in both research and application due to its simplicity and efficiency. However, communication efficiency is still a crucial issue for split learning. In this paper, we investigate multiple communication reduction methods for split learning, including cut layer size reduction, top-k sparsification, quantization, and L1 regularization. Through analysis of the cut layer size reduction and top-k sparsification, we further propose randomized top-k sparsification, to make the model generalize and converge better. This is done by selecting top-k elements with a large probability while also having a small probability to select non-top-k elements. Empirical results show that compared with other communication-reduction methods, our proposed randomized top-k sparsification achieves a better model performance under the same compression level.

Key facts. Cited by 22 papers in the map (data through 13 Oct 2026). Builds on 28 of its 55 references that are in the map. First posted on arXiv 29 May 2023.

Builds on

28 of 55 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,653
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,426
  4. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,585
  5. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  6. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,126
  7. 2019Deep Leakage from Gradientscited by 1,017
  8. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 903
  9. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 814
  10. 2015Session-based Recommendations with Recurrent Neural Networkscited by 688
  11. 2017Exploring Generalization in Deep Learningcited by 596
  12. 2020Federated Learning with Matched Averagingcited by 564
16 more
  1. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
  2. 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 394
  3. 2018Sparsified SGD with Memorycited by 376
  4. 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 311
  5. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  6. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 256
  7. 2017Sparse Communication for Distributed Gradient Descentcited by 249
  8. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 237
  9. 2018ATOMO: Communication-efficient Learning via Atomic Sparsificationcited by 136
  10. 2020Vertically Federated Graph Neural Network for Privacy-Preserving Node Classificationcited by 61
  11. 2018Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communicationcited by 51
  12. 2022Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Datacited by 36
  13. 2020When Homomorphic Encryption Marries Secret Sharing: Secure Large-Scale Sparse Logistic Regression and Applications in Risk Controlcited by 32
  14. 2022Accelerated Federated Learning with Decoupled Adaptive Optimizationcited by 23
  15. 2021Communication and Computation Reduction for Split Learning using Asynchronous Trainingcited by 11
  16. 2021Communication Optimization in Large Scale Federated Learning using Autoencoder Compressed Weight Updatescited by 1

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 6; 2026: 10.

22 papers.

Sort:

  1. 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 126
  2. 2023Communication-Efficient Split Learning via Adaptive Feature-Wise Compressioncited by 18
  3. 2024Synergizing Foundation Models and Federated Learning: A Surveycited by 14
  4. 2025Quantized-Tinyllava: a new multimodal foundation model enables efficient split learningcited by 8
  5. 2025Towards Straggler-Resilient Split Federated Learning: An Unbalanced Update Approachcited by 7
  6. 2026NSC-SL: A Bandwidth-Aware Neural Subspace Compression for Communication-Efficient Split Learningcited by 4
  7. 2023Defending Label Inference Attacks in Split Learning under Regression Settingcited by 2
  8. 2024Mask-Encoded Sparsification: Mitigating Biased Gradients in Communication-Efficient Split Learningcited by 2
  9. 2025Communication Efficient Split Learning of ViTs with Attention-based Double Compressioncited by 2
  10. 2026Lean Clients, Full Accuracy: Hybrid Zeroth- and First-Order Split Federated Learningcited by 2
  11. 2024The Robustness of Spiking Neural Networks in Communication and its Application towards Network Efficiency in Federated Learningcited by 1
  12. 2025Ampere: Communication-Efficient and High-Accuracy Split Federated Learningcited by 1
10 more
  1. 2025Oops!... They Stole it Again: Attacks on Split Learningcited by 1
  2. 2026SL-FAC: A Communication-Efficient Split Learning Framework with Frequency-Aware Compressioncited by 1
  3. 2026AutoEncoder-Compressed Parallel Split Learning for Pre-trained Model Fine-Tuningcited by 1
  4. 2025Communication and Computation Efficient Split Federated Learning in O-RANcited by 0
  5. 2026Exploiting Adaptive Channel Pruning for Communication-Efficient Split Learningcited by 0
  6. 2026LightSplit: Practical Privacy-Preserving Split Learning via Orthogonal Projectionscited by 0
  7. 2026A Privacy Study of Sparse Collaborative Inferencecited by 0
  8. 2026SplitLite: Low-Rank Residual Compression for Split Learningcited by 0
  9. 2026Contribution-Aware Bandwidth Allocation for Multimodal Split Learningcited by 0
  10. 2026Importance-Aware Feature Sparsification for Wireless Split Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Federated Split Learning with Model Pruning and Gradient Quantization in Wireless Networks
  2. 2023Communication-Efficient Federated Learning via Regularized Sparse Random Networks
  3. 2021Time-Correlated Sparsification for Communication-Efficient Federated Learning
  4. 2024FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models
  5. 2022Sparse Random Networks for Communication-Efficient Federated Learning
  6. 2024Efficient Distributed Training through Gradient Compression with Sparsification and Quantization Techniques
  7. 2025FedSparQ: Adaptive Sparse Quantization with Error Feedback for Robust & Efficient Federated Learning
  8. 2025Overcoming the Communication-Performance Tradeoff in LLM Pretraining