# LightSplit: Practical Privacy-Preserving Split Learning via Orthogonal Projections

arXiv: https://arxiv.org/abs/2605.13265 · HTML: https://mlsanity.com/paper/2605.13265/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 55 references that are in the map.
- First posted on arXiv 13 May 2026.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Inverting Gradients -- How easy is it to break privacy in federated learning?](https://mlsanity.com/paper/2003.14053/) (2020)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [iDLG: Improved Deep Leakage from Gradients](https://mlsanity.com/paper/2001.02610/) (2020)
- [FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping](https://mlsanity.com/paper/2012.13995/) (2020)
- [Compressed Sensing using Generative Models](https://mlsanity.com/paper/1703.03208/) (2017)
- [SplitFed: When Federated Learning Meets Split Learning](https://mlsanity.com/paper/2004.12088/) (2020)
- [Distributed learning of deep neural network over multiple agents](https://mlsanity.com/paper/1810.06060/) (2018)
- [RanPAC: Random Projections and Pre-trained Models for Continual Learning](https://mlsanity.com/paper/2307.02251/) (2023)
- [BottleNet++: An End-to-End Approach for Feature Compression in Device-Edge Co-Inference Systems](https://mlsanity.com/paper/1910.14315/) (2019)
- [Label Leakage and Protection in Two-party Split Learning](https://mlsanity.com/paper/2102.08504/) (2021)
- [Split Learning over Wireless Networks: Parallel Design and Resource Management](https://mlsanity.com/paper/2204.08119/) (2022)
- [NoPeek: Information leakage reduction to share activations in distributed deep learning](https://mlsanity.com/paper/2008.09161/) (2020)
- [UnSplit: Data-Oblivious Model Inversion, Model Stealing, and Label Inference Attacks Against Split Learning](https://mlsanity.com/paper/2108.09033/) (2021)
- [DISCO: Dynamic and Invariant Sensitive Channel Obfuscation for deep neural networks](https://mlsanity.com/paper/2012.11025/) (2020)
- [Reducing Communication for Split Learning by Randomized Top-k Sparsification](https://mlsanity.com/paper/2305.18469/) (2023)
- [SplitGuard: Detecting and Mitigating Training-Hijacking Attacks in Split Learning](https://mlsanity.com/paper/2108.09052/) (2021)
- [Communication-Efficient Split Learning via Adaptive Feature-Wise Compression](https://mlsanity.com/paper/2307.10805/) (2023)
- [FedVS: Straggler-Resilient and Privacy-Preserving Vertical Federated Learning for Split Models](https://mlsanity.com/paper/2304.13407/) (2023)
- [A Stealthy Wrongdoer: Feature-Oriented Reconstruction Attack against Split Learning](https://mlsanity.com/paper/2405.04115/) (2024)
- [Feature Space Hijacking Attacks against Differentially Private Split Learning](https://mlsanity.com/paper/2201.04018/) (2022)
- [Similarity-based Label Inference Attack against Training and Inference of Split Learning](https://mlsanity.com/paper/2203.05222/) (2022)
- [FrankenSplit: Efficient Neural Feature Compression with Shallow Variational Bottleneck Injection for Mobile Edge Computing](https://mlsanity.com/paper/2302.10681/) (2023)

## Abstract (arXiv)

Split learning (SL) enables collaborative training by partitioning a neural network across clients and a central server, but the cut-layer interface introduces a key challenge: high-dimensional activations incur substantial communication overhead while exposing representations vulnerable to reconstruction attacks. Existing approaches typically address efficiency or privacy in isolation, relying on additional mechanisms such as sparsification, quantization, or noise injection. We propose LightSplit, which limits information exposure and reduces communication overhead by applying a lightweight fixed orthogonal random projection at the cut layer. Based on Shannon's information theory, this projection acts as an information bottleneck that restricts instance-specific information and suppresses exploitable per-sample signals. By transmitting low-dimensional projections instead of raw activations, the server operates on lifted representations without requiring architectural modifications, ensuring compatibility with existing SL architectures. By avoiding additional trainable components on the client, the method remains lightweight and suitable for edge devices while preserving end-to-end differentiability via exact gradient propagation. As the projection is non-invertible, part of the original representation is irreversibly discarded at the client, LightSplit reduces the information available for reconstruction and limits information exposure. We extensively evaluate LightSplit on state-of-the-art benchmarks in both IID and non-IID settings across varying projection dimensions and client scales. Our results show that the method retains more than 95% of the baseline accuracy at up to 32x reduction in transmitted dimensionality while maintaining stable training dynamics.