# Automated Heterogeneous Low-Bit Quantization of Multi-Model Deep Learning Inference Pipeline

arXiv: https://arxiv.org/abs/2311.05870 · HTML: https://mlsanity.com/paper/2311.05870/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 12 references that are in the map.
- First posted on arXiv 10 Nov 2023.

## Builds on (most cited first, 8 of 8)

- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [MediaPipe Hands: On-device Real-time Hand Tracking](https://mlsanity.com/paper/2006.10214/) (2020)
- [Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors](https://mlsanity.com/paper/2006.10159/) (2020)
- [Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimization](https://mlsanity.com/paper/2110.06554/) (2021)
- [Bitwise Neural Networks](https://mlsanity.com/paper/1601.06071/) (2016)
- [Deep Neural Network Compression with Single and Multiple Level Quantization](https://mlsanity.com/paper/1803.03289/) (2018)
- [RMSMP: A Novel Deep Neural Network Quantization Framework with Row-wise Mixed Schemes and Multiple Precisions](https://mlsanity.com/paper/2111.00153/) (2021)
- [Computational Solar Energy -- Ensemble Learning Methods for Prediction of Solar Power Generation based on Meteorological Parameters in Eastern India](https://mlsanity.com/paper/2301.10159/) (2023)

## Built on by (most cited first, 1 of 1)

- [Local Masking Meets Progressive Freezing: Crafting Efficient Vision Transformers for Self-Supervised Learning](https://mlsanity.com/paper/2312.02194/) (2023)

## Abstract (arXiv)

Multiple Deep Neural Networks (DNNs) integrated into single Deep Learning (DL) inference pipelines e.g. Multi-Task Learning (MTL) or Ensemble Learning (EL), etc., albeit very accurate, pose challenges for edge deployment. In these systems, models vary in their quantization tolerance and resource demands, requiring meticulous tuning for accuracy-latency balance. This paper introduces an automated heterogeneous quantization approach for DL inference pipelines with multiple DNNs.