# Lightweight Compression of Intermediate Neural Network Features for Collaborative Intelligence

arXiv: https://arxiv.org/abs/2105.07102 · HTML: https://mlsanity.com/paper/2105.07102/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 15 May 2021.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018)
- [Data-Free Quantization Through Weight Equalization and Bias Correction](https://mlsanity.com/paper/1906.04721/) (2019)
- [ZeroQ: A Novel Zero Shot Quantization Framework](https://mlsanity.com/paper/2001.00281/) (2020)
- [Discrimination-aware Channel Pruning for Deep Neural Networks](https://mlsanity.com/paper/1810.11809/) (2018)
- [Improving Neural Network Quantization without Retraining using Outlier Channel Splitting](https://mlsanity.com/paper/1901.09504/) (2019)
- [Scalable Methods for 8-bit Training of Neural Networks](https://mlsanity.com/paper/1805.11046/) (2018)
- [JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services](https://mlsanity.com/paper/1801.08618/) (2018)
- [BottleNet: A Deep Learning Architecture for Intelligent Mobile Cloud Computing Services](https://mlsanity.com/paper/1902.01000/) (2019)
- [Value-aware Quantization for Training and Inference of Neural Networks](https://mlsanity.com/paper/1804.07802/) (2018)
- [WRPN: Wide Reduced-Precision Networks](https://mlsanity.com/paper/1709.01134/) (2017)
- [Deep feature compression for collaborative object detection](https://mlsanity.com/paper/1802.03931/) (2018)
- [Post-training 4-bit quantization of convolution networks for rapid-deployment](https://mlsanity.com/paper/1810.05723/) (2018)
- [Efficient and Accurate Approximations of Nonlinear Convolutional Networks](https://mlsanity.com/paper/1411.4229/) (2014)
- [Collaborative Intelligence: Challenges and Opportunities](https://mlsanity.com/paper/2102.06841/) (2021)
- [Near-Lossless Deep Feature Compression for Collaborative Intelligence](https://mlsanity.com/paper/1804.09963/) (2018)
- [Lightweight compression of neural network feature tensors for collaborative intelligence](https://mlsanity.com/paper/2105.06002/) (2021)
- [Back-and-Forth prediction for deep tensor compression](https://mlsanity.com/paper/2002.07036/) (2020)
- [Towards Collaborative Intelligence Friendly Architectures for Deep Learning](https://mlsanity.com/paper/1902.00147/) (2019)

## Built on by (most cited first, 7 of 7)

- [Decentralized Low-Latency Collaborative Inference via Ensembles on the Edge](https://mlsanity.com/paper/2206.03165/) (2022)
- [Semantic Edge Computing and Semantic Communications in 6G Networks: A Unifying Survey and Research Challenges](https://mlsanity.com/paper/2411.18199/) (2024)
- [Efficient Feature Compression for Machines with Global Statistics Preservation](https://mlsanity.com/paper/2512.09235/) (2025)
- [Privacy-Preserving Feature Coding for Machines](https://mlsanity.com/paper/2210.00727/) (2022)
- [Are Visual Recognition Models Robust to Image Compression?](https://mlsanity.com/paper/2304.04518/) (2023)
- [Edge-device Collaborative Computing for Multi-view Classification](https://mlsanity.com/paper/2409.15973/) (2024)
- [DFTS2: Simulating Deep Feature Transmission Over Packet Loss Channels](https://mlsanity.com/paper/2112.00794/) (2021)

## Abstract (arXiv)

In collaborative intelligence applications, part of a deep neural network (DNN) is deployed on a lightweight device such as a mobile phone or edge device, and the remaining portion of the DNN is processed where more computing resources are available, such as in the cloud. This paper presents a novel lightweight compression technique designed specifically to quantize and compress the features output by the intermediate layer of a split DNN, without requiring any retraining of the network weights. Mathematical models for estimating the clipping and quantization error of ReLU and leaky-ReLU activations at this intermediate layer are developed and used to compute optimal clipping ranges for coarse quantization. We also present a modified entropy-constrained design algorithm for quantizing clipped activations. When applied to popular object-detection and classification DNNs, we were able to compress the 32-bit floating point intermediate activations down to 0.6 to 0.8 bits, while keeping the loss in accuracy to less than 1%. When compared to HEVC, we found that the lightweight codec consistently provided better inference accuracy, by up to 1.3%. The performance and simplicity of this lightweight compression technique makes it an attractive option for coding an intermediate layer of a split neural network for edge/cloud applications.