# Slimmable Encoders for Flexible Split DNNs in Bandwidth and Resource Constrained IoT Systems

arXiv: https://arxiv.org/abs/2306.12691 · HTML: https://mlsanity.com/paper/2306.12691/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 22 Jun 2023.

## Builds on (most cited first, 23 of 23)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Slimmable Neural Networks](https://mlsanity.com/paper/1812.08928/) (2018)
- [Universally Slimmable Networks and Improved Training Techniques](https://mlsanity.com/paper/1903.05134/) (2019)
- [Density Modeling of Images using a Generalized Normalization Transformation](https://mlsanity.com/paper/1511.06281/) (2015)
- [Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges](https://mlsanity.com/paper/2103.04505/) (2021)
- [BottleNet++: An End-to-End Approach for Feature Compression in Device-Edge Co-Inference Systems](https://mlsanity.com/paper/1910.14315/) (2019)
- [Practical Full Resolution Learned Lossless Image Compression](https://mlsanity.com/paper/1811.12817/) (2018)
- [Learning Neural Network Subspaces](https://mlsanity.com/paper/2102.10472/) (2021)
- [BottleNet: A Deep Learning Architecture for Intelligent Mobile Cloud Computing Services](https://mlsanity.com/paper/1902.01000/) (2019)
- [Communication-Computation Trade-Off in Resource-Constrained Edge Inference](https://mlsanity.com/paper/2006.02166/) (2020)
- [Supervised Compression for Resource-Constrained Edge Computing Systems](https://mlsanity.com/paper/2108.11898/) (2021)
- [Collaborative Intelligence: Challenges and Opportunities](https://mlsanity.com/paper/2102.06841/) (2021)
- [Lightweight compression of neural network feature tensors for collaborative intelligence](https://mlsanity.com/paper/2105.06002/) (2021)
- [Back-and-Forth prediction for deep tensor compression](https://mlsanity.com/paper/2002.07036/) (2020)
- [SplitNets: Designing Neural Architectures for Efficient Distributed Computing on Head-Mounted Systems](https://mlsanity.com/paper/2204.04705/) (2022)
- [Single-Training Collaborative Object Detectors Adaptive to Bandwidth and Computation](https://mlsanity.com/paper/2105.00591/) (2021)
- [LCS: Learning Compressible Subspaces for Adaptive Network Compression at Inference Time](https://mlsanity.com/paper/2110.04252/) (2021)
- [Compressing Representations for Embedded Deep Learning](https://mlsanity.com/paper/1911.10321/) (2019)

## Built on by (most cited first, 3 of 3)

- [Semantic Edge Computing and Semantic Communications in 6G Networks: A Unifying Survey and Research Challenges](https://mlsanity.com/paper/2411.18199/) (2024)
- [NaviSplit: Dynamic Multi-Branch Split DNNs for Efficient Distributed Autonomous Navigation](https://mlsanity.com/paper/2406.13086/) (2024)
- [CADENCE: Context-Adaptive Depth Estimation for Navigation and Computational Efficiency](https://mlsanity.com/paper/2604.07286/) (2026)

## Abstract (arXiv)

The execution of large deep neural networks (DNN) at mobile edge devices requires considerable consumption of critical resources, such as energy, while imposing demands on hardware capabilities. In approaches based on edge computing the execution of the models is offloaded to a compute-capable device positioned at the edge of 5G infrastructures. The main issue of the latter class of approaches is the need to transport information-rich signals over wireless links with limited and time-varying capacity. The recent split computing paradigm attempts to resolve this impasse by distributing the execution of DNN models across the layers of the systems to reduce the amount of data to be transmitted while imposing minimal computing load on mobile devices. In this context, we propose a novel split computing approach based on slimmable ensemble encoders. The key advantage of our design is the ability to adapt computational load and transmitted data size in real-time with minimal overhead and time. This is in contrast with existing approaches, where the same adaptation requires costly context switching and model loading. Moreover, our model outperforms existing solutions in terms of compression efficacy and execution time, especially in the context of weak mobile devices. We present a comprehensive comparison with the most advanced split computing solutions, as well as an experimental evaluation on GPU-less devices.