mlsanityData to 7 Oct 2026

HGQ: High Granularity Quantization for Real-time Neural Networks on FPGAs

arXiv 2405.00645 ↗First posted 1 May 2024cs.LG physics.ins-detcited by 9

Not on the map. Why?

Abstract (arXiv)

Neural networks with sub-microsecond inference latency are required by many critical applications. Targeting such applications deployed on FPGAs, we present High Granularity Quantization (HGQ), a quantization-aware training framework that optimizes parameter bit-widths through gradient descent. Unlike conventional methods, HGQ determines the optimal bit-width for each parameter independently, making it suitable for hardware platforms supporting heterogeneous arbitrary precision arithmetic. In our experiments, HGQ shows superior performance compared to existing network compression methods, achieving orders of magnitude reduction in resource consumption and latency while maintaining the accuracy on several benchmark tasks. These improvements enable the deployment of complex models previously infeasible due to resource or latency constraints. HGQ is open-source and is used for developing next-generation trigger systems at the CERN ATLAS and CMS experiments for particle physics, enabling the use of advanced machine learning models for real-time data selection with sub-microsecond latency.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 39 of its 100 references that are in the map. First posted on arXiv 1 May 2024.

Builds on

39 of 100 references.

References · 39

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  4. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  5. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  6. 2019Learned Step Size Quantizationcited by 604
  7. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  8. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  9. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  10. 2019PyHessian: Neural Networks Through the Lens of the Hessiancited by 291
  11. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  12. 2016Trained Ternary Quantizationcited by 203
27 more
  1. 2023TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddingscited by 198
  2. 2021Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Bettercited by 153
  3. 2017Towards Accurate Binary Convolutional Neural Networkcited by 148
  4. 2016FINN: A Framework for Fast, Scalable Binarized Neural Network Inferencecited by 141
  5. 2018Fast inference of deep neural networks in FPGAs for particle physicscited by 132
  6. 2020Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectorscited by 78
  7. 2019AutoQ: Automated Kernel-Wise Neural Network Quantizationcited by 60
  8. 2018FINN-R: An End-to-End Deep-Learning Framework for Fast Exploration of Quantized Neural Networkscited by 54
  9. 2018Value-aware Quantization for Training and Inference of Neural Networkscited by 49
  10. 2021Differentiable Model Compression via Pseudo Quantization Noisecited by 44
  11. 2021Fast convolutional neural networks on FPGAs with hls4mlcited by 42
  12. 2021Applications and Techniques for Fast Machine Learning in Sciencecited by 32
  13. 2019Adaptive Loss-aware Quantization for Multi-bit Networkscited by 29
  14. 2020Mix and Match: A Novel FPGA-Centric Deep Neural Network Quantization Frameworkcited by 27
  15. 2017Network Sketching: Exploiting Binary Structure in Deep CNNscited by 22
  16. 2024NeuraLUT: Hiding Neural Network Density in Boolean Synthesizable Functionscited by 21
  17. 2023Real-time Neural Network Inference on Extremely Weak Devices: Agile Offloading with Explainable AIcited by 17
  18. 2024StreamVC: Real-Time Low-Latency Voice Conversioncited by 15
  19. 2022LL-GNN: Low Latency Graph Neural Networks on FPGAs for High Energy Physicscited by 12
  20. 2024Ultrafast jet classification on FPGAs for the HL-LHCcited by 12
  21. 2021GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsitycited by 10
  22. 2020Hardware-Centric AutoML for Mixed-Precision Quantizationcited by 8
  23. 2019Unrolling Ternary Neural Networkscited by 7
  24. 2023FPGA Resource-aware Structured Pruning for Real-Time Neural Networkscited by 7
  25. 2024SymbolNet: Neural Symbolic Regression with Adaptive Dynamic Pruning for Compressioncited by 6
  26. 2024A2Q+: Improving Accumulator-Aware Weight Quantizationcited by 5
  27. 2023MetaML: Automating Customizable Cross-Stage Design-Flow for Deep Learning Accelerationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 5.

9 papers.

  1. 2025hls4ml: A Flexible, Open-Source Platform for Deep Learning Acceleration on Reconfigurable Hardwarecited by 8
  2. 2025WARP-LUTs -- Walsh-Assisted Relaxation for Probabilistic Look Up Tablescited by 6
  3. 2025MetaML-Pro: Cross-Stage Design Flow Automation for Efficient Deep Learning Accelerationcited by 4
  4. 2026JetFormer: A Scalable and Efficient Transformer for Jet Tagging from Offline Analysis to FPGA Triggerscited by 3
  5. 2025BitHEP -- The Limits of Low-Precision ML in HEPcited by 1
  6. 2026HGQ-LUT: Fast LUT-Aware Training and Efficient Architectures for DNN Inferencecited by 1
  7. 2026PQuantML: A Tool for End-to-End Hardware-aware Model Compressioncited by 0
  8. 2026SciFi: A Safe, Lightweight, User-Friendly, and Fully Autonomous Agentic AI Workflow for Scientific Applicationscited by 0
  9. 2026FQTree: Fine-grained Quantization and Hardware Generation of Boosted Decision Treescited by 0