# Dynamic Encoding and Decoding of Information for Split Learning in Mobile-Edge Computing: Leveraging Information Bottleneck Theory

arXiv: https://arxiv.org/abs/2309.02787 · HTML: https://mlsanity.com/paper/2309.02787/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 6 Sep 2023.

## Builds on (most cited first, 19 of 19)

- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [Emergence of Invariance and Disentanglement in Deep Representations](https://mlsanity.com/paper/1706.01350/) (2017)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Convergence of Edge Computing and Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/1907.08349/) (2019)
- [6G Networks: Beyond Shannon Towards Semantic and Goal-Oriented Communications](https://mlsanity.com/paper/2011.14844/) (2020)
- [Wireless Network Intelligence at the Edge](https://mlsanity.com/paper/1812.02858/) (2018)
- [Semantic Communications: Principles and Challenges](https://mlsanity.com/paper/2201.01389/) (2021)
- [Split Learning over Wireless Networks: Parallel Design and Resource Management](https://mlsanity.com/paper/2204.08119/) (2022)
- [Split Learning for collaborative deep learning in healthcare](https://mlsanity.com/paper/1912.12115/) (2019)
- [Nonlinear Information Bottleneck](https://mlsanity.com/paper/1705.02436/) (2017)
- [Estimating Mixture Entropy with Pairwise Distances](https://mlsanity.com/paper/1706.02419/) (2017)
- [Caveats for information bottleneck in deterministic scenarios](https://mlsanity.com/paper/1808.07593/) (2018)
- [Adaptive Estimators Show Information Compression in Deep Neural Networks](https://mlsanity.com/paper/1902.09037/) (2019)
- [The Dual Information Bottleneck](https://mlsanity.com/paper/2006.04641/) (2020)
- [Semantic Information Recovery in Wireless Networks](https://mlsanity.com/paper/2204.13366/) (2022)
- [Goal-Oriented Communication for Edge Learning based on the Information Bottleneck](https://mlsanity.com/paper/2202.12639/) (2022)

## Built on by (most cited first, 1 of 1)

- [LPUF-AuthNet: A Lightweight PUF-Based IoT Authentication via Tandem Neural Networks and Split Learning](https://mlsanity.com/paper/2410.12190/) (2024)

## Abstract (arXiv)

Split learning is a privacy-preserving distributed learning paradigm in which an ML model (e.g., a neural network) is split into two parts (i.e., an encoder and a decoder). The encoder shares so-called latent representation, rather than raw data, for model training. In mobile-edge computing, network functions (such as traffic forecasting) can be trained via split learning where an encoder resides in a user equipment (UE) and a decoder resides in the edge network. Based on the data processing inequality and the information bottleneck (IB) theory, we present a new framework and training mechanism to enable a dynamic balancing of the transmission resource consumption with the informativeness of the shared latent representations, which directly impacts the predictive performance. The proposed training mechanism offers an encoder-decoder neural network architecture featuring multiple modes of complexity-relevance tradeoffs, enabling tunable performance. The adaptability can accommodate varying real-time network conditions and application requirements, potentially reducing operational expenditure and enhancing network agility. As a proof of concept, we apply the training mechanism to a millimeter-wave (mmWave)-enabled throughput prediction problem. We also offer new insights and highlight some challenges related to recurrent neural networks from the perspective of the IB theory. Interestingly, we find a compression phenomenon across the temporal domain of the sequential model, in addition to the compression phase that occurs with the number of training epochs.