# Trainable Bitwise Soft Quantization for Input Feature Compression

arXiv: https://arxiv.org/abs/2603.05172 · HTML: https://mlsanity.com/paper/2603.05172/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 44 references that are in the map.
- First posted on arXiv 5 Mar 2026.

## Builds on (most cited first, 24 of 24)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [8-bit Optimizers via Block-wise Quantization](https://mlsanity.com/paper/2110.02861/) (2021)
- [Lossy Image Compression with Compressive Autoencoders](https://mlsanity.com/paper/1703.00395/) (2017)
- [Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks](https://mlsanity.com/paper/1908.05033/) (2019)
- [Quantization Networks](https://mlsanity.com/paper/1911.09464/) (2019)
- [Improving Neural Network Quantization without Retraining using Outlier Channel Splitting](https://mlsanity.com/paper/1901.09504/) (2019)
- [SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud](https://mlsanity.com/paper/2008.06402/) (2020)
- [LassoNet: A Neural Network with Feature Sparsity](https://mlsanity.com/paper/1907.12207/) (2019)
- [OpenML-Python: an extensible Python API for OpenML](https://mlsanity.com/paper/1911.02490/) (2019)
- [Estimating or Propagating Gradients Through Stochastic Neurons](https://mlsanity.com/paper/1305.2982/) (2013)
- [End-to-end Learning of Compressible Features](https://mlsanity.com/paper/2007.11797/) (2020)
- [Supervised Compression for Resource-Constrained Edge Computing Systems](https://mlsanity.com/paper/2108.11898/) (2021)
- [Auto-Split: A General Framework of Collaborative Edge-Cloud AI](https://mlsanity.com/paper/2108.13041/) (2021)
- [BottleFit: Learning Compressed Representations in Deep Neural Networks for Effective and Efficient Split Computing](https://mlsanity.com/paper/2201.02693/) (2022)
- [Multi-task learning with compressible features for Collaborative Intelligence](https://mlsanity.com/paper/1902.05179/) (2019)
- [Collaborative Intelligence: Challenges and Opportunities](https://mlsanity.com/paper/2102.06841/) (2021)
- [Lightweight compression of neural network feature tensors for collaborative intelligence](https://mlsanity.com/paper/2105.06002/) (2021)
- [Widening Access to Applied Machine Learning with TinyML](https://mlsanity.com/paper/2106.04008/) (2021)
- [Pruning Random Forests for Prediction on a Budget](https://mlsanity.com/paper/1606.05060/) (2016)
- [Feature Compression for Rate Constrained Object Detection on the Edge](https://mlsanity.com/paper/2204.07314/) (2022)
- [Input Selection for Bandwidth-Limited Neural Network Inference](https://mlsanity.com/paper/1906.04673/) (2019)

## Abstract (arXiv)

The growing demand for machine learning applications in the context of the Internet of Things calls for new approaches to optimize the use of limited compute and memory resources. Despite significant progress that has been made w.r.t. reducing model sizes and improving efficiency, many applications still require remote servers to provide the required resources. However, such approaches rely on transmitting data from edge devices to remote servers, which may not always be feasible due to bandwidth, latency, or energy constraints. We propose a task-specific, trainable feature quantization layer that compresses the input features of a neural network. This can significantly reduce the amount of data that needs to be transferred from the device to a remote server. In particular, the layer allows each input feature to be quantized to a user-defined number of bits, enabling a simple on-device compression at the time of data collection. The layer is designed to approximate step functions with sigmoids, enabling trainable quantization thresholds. By concatenating outputs from multiple sigmoids, introduced as bitwise soft quantization, it achieves trainable quantized values when integrated with a neural network. We compare our method to full-precision inference as well as to several quantization baselines. Experiments show that our approach outperforms standard quantization methods, while maintaining accuracy levels close to those of full-precision models. In particular, depending on the dataset, compression factors of 5× to 16× can be achieved compared to 32-bit input without significant performance loss.