# Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors

arXiv: https://arxiv.org/abs/2006.10159 · HTML: https://mlsanity.com/paper/2006.10159/

## Key facts

- Cited by 80 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2020.

## Built on by (most cited first, 30 of 80)

- [Fast convolutional neural networks on FPGAs with hls4ml](https://mlsanity.com/paper/2101.05108/) (2021)
- [Applications and Techniques for Fast Machine Learning in Science](https://mlsanity.com/paper/2110.13041/) (2021)
- [Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference](https://mlsanity.com/paper/2102.11289/) (2021)
- [PolyLUT: Learning Piecewise Polynomials for Ultra-Low Latency FPGA LUT-based Inference](https://mlsanity.com/paper/2309.02334/) (2023)
- [Machine Learning for Microcontroller-Class Hardware: A Review](https://mlsanity.com/paper/2205.14550/) (2022)
- [PELICAN: Permutation Equivariant and Lorentz Invariant or Covariant Aggregator Network for Particle Physics](https://mlsanity.com/paper/2211.00454/) (2022)
- [NeuraLUT: Hiding Neural Network Density in Boolean Synthesizable Functions](https://mlsanity.com/paper/2403.00849/) (2024)
- [Charged particle tracking via edge-classifying interaction networks](https://mlsanity.com/paper/2103.16701/) (2021)
- [Graph Neural Networks for Charged Particle Tracking on FPGAs](https://mlsanity.com/paper/2112.02048/) (2021)
- [Online-compatible Unsupervised Non-resonant Anomaly Detection](https://mlsanity.com/paper/2111.06417/) (2021)
- [Graph Neural Networks in Particle Physics: Implementations, Innovations, and Challenges](https://mlsanity.com/paper/2203.12852/) (2022)
- [FastML Science Benchmarks: Accelerating Real-Time Scientific Edge Machine Learning](https://mlsanity.com/paper/2207.07958/) (2022)
- [LL-GNN: Low Latency Graph Neural Networks on FPGAs for High Energy Physics](https://mlsanity.com/paper/2209.14065/) (2022)
- [Ultrafast jet classification on FPGAs for the HL-LHC](https://mlsanity.com/paper/2402.01876/) (2024)
- [PolyLUT-Add: FPGA-based LUT Inference with Wide Inputs](https://mlsanity.com/paper/2406.04910/) (2024)
- [Real-time semantic segmentation on FPGAs for autonomous vehicles with hls4ml](https://mlsanity.com/paper/2205.07690/) (2022)
- [FIT: A Metric for Model Sensitivity](https://mlsanity.com/paper/2210.08502/) (2022)
- [QONNX: Representing Arbitrary-Precision Quantized Neural Networks](https://mlsanity.com/paper/2206.07527/) (2022)
- [Nanosecond machine learning event classification with boosted decision trees in FPGA for high energy physics](https://mlsanity.com/paper/2104.03408/) (2021)
- [Symbolic Regression on FPGAs for Fast Machine Learning Inference](https://mlsanity.com/paper/2305.04099/) (2023)
- [Sustainable Supercomputing for AI: GPU Power Capping at HPC Scale](https://mlsanity.com/paper/2402.18593/) (2024)
- [HGQ: High Granularity Quantization for Real-time Neural Networks on FPGAs](https://mlsanity.com/paper/2405.00645/) (2024)
- [hls4ml: A Flexible, Open-Source Platform for Deep Learning Acceleration on Reconfigurable Hardware](https://mlsanity.com/paper/2512.01463/) (2025)
- [Within-Camera Multilayer Perceptron DVS Denoising](https://mlsanity.com/paper/2304.07543/) (2023)
- [PolyLUT: Ultra-low Latency Polynomial Inference with Hardware-Aware Structured Pruning](https://mlsanity.com/paper/2501.08043/) (2025)
- [End-to-end workflow for machine learning-based qubit readout with QICK and hls4ml](https://mlsanity.com/paper/2501.14663/) (2025)
- [da4ml: Distributed Arithmetic for Real-time Neural Networks on FPGAs](https://mlsanity.com/paper/2507.04535/) (2025)
- [Reducing Computational Complexity of Neural Networks in Optical Channel Equalization: From Concepts to Implementation](https://mlsanity.com/paper/2208.12866/) (2022)
- [FPGA Resource-aware Structured Pruning for Real-Time Neural Networks](https://mlsanity.com/paper/2308.05170/) (2023)
- [SymbolNet: Neural Symbolic Regression with Adaptive Dynamic Pruning for Compression](https://mlsanity.com/paper/2401.09949/) (2024)

## Abstract (arXiv)

Although the quest for more accurate solutions is pushing deep learning research towards larger and more complex algorithms, edge devices demand efficient inference and therefore reduction in model size, latency and energy consumption. One technique to limit model size is quantization, which implies using fewer bits to represent weights and biases. Such an approach usually results in a decline in performance. Here, we introduce a method for designing optimally heterogeneously quantized versions of deep neural network models for minimum-energy, high-accuracy, nanosecond inference and fully automated deployment on chip. With a per-layer, per-parameter type automatic quantization procedure, sampling from a wide range of quantizers, model energy consumption and size are minimized while high accuracy is maintained. This is crucial for the event selection procedure in proton-proton collisions at the CERN Large Hadron Collider, where resources are strictly limited and a latency of mathcal O(1)~μs is required. Nanosecond inference and a resource consumption reduced by a factor of 50 when implemented on field-programmable gate array hardware are achieved.