mlsanityData to 14 Oct 2026

HAQ: Hardware-Aware Automated Quantization with Mixed Precision

arXiv 1811.08886 ↗First posted 21 Nov 2018cs.CVcited by 443

Not on the map. Why?

Abstract (arXiv)

Model quantization is a widely used technique to compress and accelerate deep neural network (DNN) inference. Emergent DNN hardware accelerators begin to support mixed precision (1-8 bits) to further improve the computation efficiency, which raises a great challenge to find the optimal bitwidth for each layer: it requires domain experts to explore the vast design space trading off among accuracy, latency, energy, and model size, which is both time-consuming and sub-optimal. Conventional quantization algorithm ignores the different hardware architectures and quantizes all the layers in a uniform way. In this paper, we introduce the Hardware-Aware Automated Quantization (HAQ) framework which leverages the reinforcement learning to automatically determine the quantization policy, and we take the hardware accelerator's feedback in the design loop. Rather than relying on proxy signals such as FLOPs and model size, we employ a hardware simulator to generate direct feedback signals (latency and energy) to the RL agent. Compared with conventional methods, our framework is fully automated and can specialize the quantization policy for different neural network architectures and hardware architectures. Our framework effectively reduced the latency by 1.4-1.95x and the energy consumption by 1.9x with negligible loss of accuracy compared with the fixed bitwidth (8 bits) quantization. Our framework reveals that the optimal policies on different hardware architectures (i.e., edge and cloud architectures) under different resource constraints (i.e., latency, energy and model size) are drastically different. We interpreted the implication of different quantization policies, which offer insights for both neural network architecture design and hardware architecture design.

Key facts. Cited by 443 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 95; 2022: 74; 2023: 77; 2024: 61; 2025: 77; 2026: 59.

443 papers.

Sort:

By area:

  1. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  2. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  3. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  4. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  5. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  6. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  7. 2021Post-Training Quantization for Vision Transformercited by 278
  8. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 263
  9. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  10. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  11. 2021Adaptive Token Sampling For Efficient Vision Transformerscited by 183
  12. 2021Implicit Neural Representations for Image Compressioncited by 162
38 more
  1. 2022On-Device Training Under 256KB Memorycited by 160
  2. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 141
  3. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 137
  4. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  5. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generationcited by 115
  6. 2022I-ViT: Integer-only Quantization for Efficient Vision Transformer Inferencecited by 95
  7. 2024ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identificationcited by 93
  8. 2023OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantizationcited by 92
  9. 2021MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learningcited by 89
  10. 2021Network Quantization with Element-wise Gradient Scalingcited by 87
  11. 2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
  12. 2024AffineQuant: Affine Transformation Quantization for Large Language Modelscited by 67
  13. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 65
  14. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 64
  15. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimizationcited by 61
  16. 2023SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformercited by 60
  17. 2023MixFormerV2: Efficient Fully Transformer Trackingcited by 57
  18. 2021Anycost GANs for Interactive Image Synthesis and Editingcited by 54
  19. 2022Mixed-Precision Neural Network Quantization via Learned Layer-wise Importancecited by 54
  20. 2022ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantizationcited by 51
  21. 2022Revisiting Random Channel Pruning for Neural Network Compressioncited by 50
  22. 2023Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMscited by 50
  23. 2025Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Modelscited by 46
  24. 2021Differentiable Model Compression via Pseudo Quantization Noisecited by 45
  25. 2022Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Modelscited by 45
  26. 2024MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantizationcited by 43
  27. 2021MQBench: Towards Reproducible and Deployable Model Quantization Benchmarkcited by 40
  28. 2024Tiny Machine Learning: Progress and Futurescited by 40
  29. 2021BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantizationcited by 39
  30. 2021OMPQ: Orthogonal Mixed Precision Quantizationcited by 39
  31. 2021Dynamic Network Quantization for Efficient Video Inferencecited by 36
  32. 2022NIPQ: Noise proxy-based Integrated Pseudo-Quantizationcited by 35
  33. 2023A Survey of Techniques for Optimizing Transformer Inferencecited by 35
  34. 2021Applications and Techniques for Fast Machine Learning in Sciencecited by 32
  35. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
  36. 2021Ps and Qs: Quantization-aware pruning for efficient low latency neural network inferencecited by 31
  37. 2021DKM: Differentiable K-Means Clustering Layer for Neural Network Compressioncited by 31
  38. 2021Auto-Split: A General Framework of Collaborative Edge-Cloud AIcited by 30

50 most cited of 443 All 443 (JSON)