mlsanityData to 7 Oct 2026

LightSplit: Practical Privacy-Preserving Split Learning via Orthogonal Projections

arXiv 2605.13265 ↗First posted 13 May 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Split learning (SL) enables collaborative training by partitioning a neural network across clients and a central server, but the cut-layer interface introduces a key challenge: high-dimensional activations incur substantial communication overhead while exposing representations vulnerable to reconstruction attacks. Existing approaches typically address efficiency or privacy in isolation, relying on additional mechanisms such as sparsification, quantization, or noise injection. We propose LightSplit, which limits information exposure and reduces communication overhead by applying a lightweight fixed orthogonal random projection at the cut layer. Based on Shannon's information theory, this projection acts as an information bottleneck that restricts instance-specific information and suppresses exploitable per-sample signals. By transmitting low-dimensional projections instead of raw activations, the server operates on lifted representations without requiring architectural modifications, ensuring compatibility with existing SL architectures. By avoiding additional trainable components on the client, the method remains lightweight and suitable for edge devices while preserving end-to-end differentiability via exact gradient propagation. As the projection is non-invertible, part of the original representation is irreversibly discarded at the client, LightSplit reduces the information available for reconstruction and limits information exposure. We extensively evaluate LightSplit on state-of-the-art benchmarks in both IID and non-IID settings across varying projection dimensions and client scales. Our results show that the method retains more than 95% of the baseline accuracy at up to 32x reduction in transmitted dimensionality while maintaining stable training dynamics.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 55 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

34 of 55 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  7. 2019Deep Leakage from Gradientscited by 1,004
  8. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  9. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  10. 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 391
  11. 2020iDLG: Improved Deep Leakage from Gradientscited by 367
  12. 2017Compressed Sensing using Generative Modelscited by 337
22 more
  1. 2020FLTrust: Byzantine-robust Federated Learning via Trust Bootstrappingcited by 330
  2. 2020SplitFed: When Federated Learning Meets Split Learningcited by 308
  3. 2018Distributed learning of deep neural network over multiple agentscited by 277
  4. 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 162
  5. 2019BottleNet++: An End-to-End Approach for Feature Compression in Device-Edge Co-Inference Systemscited by 94
  6. 2021Label Leakage and Protection in Two-party Split Learningcited by 87
  7. 2022Split Learning over Wireless Networks: Parallel Design and Resource Managementcited by 74
  8. 2020NoPeek: Information leakage reduction to share activations in distributed deep learningcited by 66
  9. 2021UnSplit: Data-Oblivious Model Inversion, Model Stealing, and Label Inference Attacks Against Split Learningcited by 52
  10. 2020DISCO: Dynamic and Invariant Sensitive Channel Obfuscation for deep neural networkscited by 23
  11. 2023Reducing Communication for Split Learning by Randomized Top-k Sparsificationcited by 22
  12. 2021SplitGuard: Detecting and Mitigating Training-Hijacking Attacks in Split Learningcited by 20
  13. 2023Communication-Efficient Split Learning via Adaptive Feature-Wise Compressioncited by 18
  14. 2023FedVS: Straggler-Resilient and Privacy-Preserving Vertical Federated Learning for Split Modelscited by 17
  15. 2024A Stealthy Wrongdoer: Feature-Oriented Reconstruction Attack against Split Learningcited by 17
  16. 2022Similarity-based Label Inference Attack against Training and Inference of Split Learningcited by 15
  17. 2022Feature Space Hijacking Attacks against Differentially Private Split Learningcited by 14
  18. 2023FrankenSplit: Efficient Neural Feature Compression with Shallow Variational Bottleneck Injection for Mobile Edge Computingcited by 12
  19. 2022C3-SL: Circular Convolution-Based Batch-Wise Compression for Communication-Efficient Split Learningcited by 8
  20. 2025SafeSplit: A Novel Defense Against Client-Side Backdoor Attacks in Split Learning (Full Version)cited by 7
  21. 2023EcoFed: Efficient Communication for DNN Partitioning-based Federated Learningcited by 5
  22. 2024Mask-Encoded Sparsification: Mitigating Biased Gradients in Communication-Efficient Split Learningcited by 2

Built on by

Not cited yet.