mlsanityData to 14 Oct 2026

Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search

arXiv 1812.00090 ↗First posted 30 Nov 2018cs.CVcited by 125

Not on the map. Why?

Abstract (arXiv)

Recent work in network quantization has substantially reduced the time and space complexity of neural network inference, enabling their deployment on embedded and mobile devices with limited computational and memory resources. However, existing quantization methods often represent all weights and activations with the same precision (bit-width). In this paper, we explore a new dimension of the design space: quantizing different layers with different bit-widths. We formulate this problem as a neural architecture search problem and propose a novel differentiable neural architecture search (DNAS) framework to efficiently explore its exponential search space with gradient-based optimization. Experiments show we surpass the state-of-the-art compression of ResNet on CIFAR-10 and ImageNet. Our quantized models with 21.1x smaller model size or 103.9x lower computational cost can still outperform baseline quantized or even full precision models.

Key facts. Cited by 125 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 42; 2022: 19; 2023: 25; 2024: 15; 2025: 14; 2026: 10.

125 papers.

Sort:

By area:

  1. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  2. 2021I-BERT: Integer-only BERT Quantizationcited by 255
  3. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 141
  4. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 80
  5. 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 79
  6. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 65
  7. 2021VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inferencecited by 64
  8. 2021A Comprehensive Survey on Hardware-Aware Neural Architecture Searchcited by 61
  9. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimizationcited by 61
  10. 2022Mixed-Precision Neural Network Quantization via Learned Layer-wise Importancecited by 54
  11. 2021MQBench: Towards Reproducible and Deployable Model Quantization Benchmarkcited by 40
  12. 2021Dynamic Network Quantization for Efficient Video Inferencecited by 36
38 more
  1. 2021Applications and Techniques for Fast Machine Learning in Sciencecited by 32
  2. 2021Auto-Split: A General Framework of Collaborative Edge-Cloud AIcited by 30
  3. 2022SDQ: Stochastic Differentiable Quantization with Mixed Precisioncited by 28
  4. 2021Sharpness-aware Quantization for Deep Neural Networkscited by 25
  5. 2023A Practical Mixed Precision Algorithm for Post-Training Quantizationcited by 24
  6. 2021Bit-Mixer: Mixed-precision networks with runtime bit-width selectioncited by 22
  7. 2022Make RepVGG Greater Again: A Quantization-aware Approachcited by 22
  8. 2023EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantizationcited by 22
  9. 2021PSViT: Better Vision Transformer via Token Pooling and Attention Sharingcited by 21
  10. 2023Post-training Quantization for Text-to-Image Diffusion Models with Progressive Calibration and Activation Relaxingcited by 21
  11. 2021BatchQuant: Quantized-for-all Architecture Search with Robust Quantizercited by 20
  12. 2023Data Quality-aware Mixed-precision Quantization via Hybrid Reinforcement Learningcited by 17
  13. 2021HAO: Hardware-aware neural Architecture Optimization for Efficient Inferencecited by 16
  14. 2021Automatic Mixed-Precision Quantization Search of BERTcited by 16
  15. 2023Patch-wise Mixed-Precision Quantization of Vision Transformercited by 15
  16. 2021Automated Deep Learning: Neural Architecture Search Is Not the Endcited by 13
  17. 2024Retraining-free Model Quantization via One-Shot Weight-Coupling Learningcited by 13
  18. 2021Dynamic Precision Analog Computing for Neural Networkscited by 12
  19. 2021Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Acceleratorscited by 12
  20. 2021LANA: Latency Aware Network Accelerationcited by 11
  21. 2021BMPQ: Bit-Gradient Sensitivity Driven Mixed-Precision Quantization of DNNs from Scratchcited by 11
  22. 2022FIT: A Metric for Model Sensitivitycited by 11
  23. 2023Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devicescited by 11
  24. 2024Channel-Wise Mixed-Precision Quantization for Large Language Modelscited by 11
  25. 2025FlexGaussian: Flexible and Cost-Effective Training-Free Compression for 3D Gaussian Splattingcited by 11
  26. 2021Confounding Tradeoffs for Neural Network Quantizationcited by 9
  27. 2022A Silicon Photonic Accelerator for Convolutional Neural Networks with Heterogeneous Quantizationcited by 9
  28. 2022Bitwidth-Adaptive Quantization-Aware Neural Network Training: A Meta-Learning Approachcited by 9
  29. 2025FGMP: Fine-Grained Mixed-Precision Weight and Activation Quantization for Hardware-Accelerated LLM Inferencecited by 9
  30. 2023Revisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference?cited by 8
  31. 2025Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deploymentcited by 8
  32. 2021Integer-arithmetic-only Certified Robustness for Quantized Neural Networkscited by 7
  33. 2022Mixed-Precision Neural Networks: A Surveycited by 7
  34. 2023Mixed-Precision Quantization for Deep Vision Models with Integer Quadratic Programmingcited by 7
  35. 2023MetaMix: Meta-state Precision Searcher for Mixed-precision Activation Quantizationcited by 7
  36. 2024Content-Aware Radiance Fields: Aligning Model Complexity with Scene Intricacy Through Learned Bitwidth Quantizationcited by 7
  37. 2021Single-path Bit Sharing for Automatic Loss-aware Model Compressioncited by 6
  38. 2021Effective and Fast: A Novel Sequential Single Path Search for Mixed-Precision Quantizationcited by 6

50 most cited of 125 All 125 (JSON)