mlsanityData to 7 Oct 2026

EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantization

arXiv 2307.10554 ↗First posted 20 Jul 2023cs.CV cs.AIcited by 22

Not on the map. Why?

Abstract (arXiv)

Mixed-Precision Quantization~(MQ) can achieve a competitive accuracy-complexity trade-off for models. Conventional training-based search methods require time-consuming candidate training to search optimized per-layer bit-width configurations in MQ. Recently, some training-free approaches have presented various MQ proxies and significantly improve search efficiency. However, the correlation between these proxies and quantization accuracy is poorly understood. To address the gap, we first build the MQ-Bench-101, which involves different bit configurations and quantization results. Then, we observe that the existing training-free proxies perform weak correlations on the MQ-Bench-101. To efficiently seek superior proxies, we develop an automatic search of proxies framework for MQ via evolving algorithms. In particular, we devise an elaborate search space involving the existing proxies and perform an evolution search to discover the best correlated MQ proxy. We proposed a diversity-prompting selection strategy and compatibility screening protocol to avoid premature convergence and improve search efficiency. In this way, our Evolving proxies for Mixed-precision Quantization~(EMQ) framework allows the auto-generation of proxies without heavy tuning and expert knowledge. Extensive experiments on ImageNet with various ResNet and MobileNet families demonstrate that our EMQ obtains superior performance than state-of-the-art mixed-precision methods at a significantly reduced cost. The code will be released.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 46 of its 60 references that are in the map. First posted on arXiv 20 Jul 2023.

Builds on

46 of 60 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  4. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  5. 2020Knowledge Distillation: A Surveycited by 1,578
  6. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  7. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  8. 2019Learned Step Size Quantizationcited by 604
  9. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  10. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 475
  11. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 467
  12. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
34 more
  1. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  2. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  3. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
  4. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  5. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  6. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 265
  7. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 250
  8. 2020Neural Architecture Search without Trainingcited by 241
  9. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  10. 2020AutoML-Zero: Evolving Machine Learning Algorithms From Scratchcited by 137
  11. 2018Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Searchcited by 122
  12. 2020APQ: Joint Search for Network Architecture, Pruning and Quantization Policycited by 98
  13. 2018Faster gaze prediction with dense networks and Fisher pruningcited by 92
  14. 2020Rethinking Differentiable Search for Mixed-Precision Neural Networkscited by 79
  15. 2021Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognitioncited by 71
  16. 2019AdaBits: Neural Network Quantization with Adaptive Bit-Widthscited by 63
  17. 2019AutoQ: Automated Kernel-Wise Neural Network Quantizationcited by 60
  18. 2020HAWQV3: Dyadic Neural Network Quantizationcited by 59
  19. 2020FracBits: Mixed Precision Quantization via Fractional Bit-Widthscited by 53
  20. 2018Value-aware Quantization for Training and Inference of Neural Networkscited by 49
  21. 2022Exploring Inter-Channel Correlation for Diversity-preserved KnowledgeDistillationcited by 44
  22. 2023NORM: Knowledge Distillation via N-to-One Representation Matchingcited by 44
  23. 2020HMQ: Hardware Friendly Mixed Precision Quantization Block for CNNscited by 43
  24. 2021OMPQ: Orthogonal Mixed Precision Quantizationcited by 37
  25. 2020Search What You Want: Barrier Panelty NAS for Mixed Precision Quantizationcited by 31
  26. 2023DisWOT: Student Architecture Search for Distillation WithOut Trainingcited by 29
  27. 2021Selfish Sparse RNN Trainingcited by 26
  28. 2021EPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Searchcited by 24
  29. 2021Loss Function Discovery for Object Detection via Convergence-Simulation Driven Searchcited by 20
  30. 2021AutoLoss-Zero: Searching Loss Functions from Scratch for Generic Taskscited by 19
  31. 2021MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detectioncited by 16
  32. 2020One Weight Bitwidth to Rule Them Allcited by 13
  33. 2023Catch-Up Distillation: You Only Need to Train Once for Accelerating Samplingcited by 13
  34. 2023RD-NAS: Enhancing One-shot Supernet Ranking Ability via Ranking Distillation from Zero-cost Proxiescited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 8; 2025: 7; 2026: 2.

22 papers.

Sort:

By area:

  1. 2024Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Modelscited by 40
  2. 2024STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMscited by 31
  3. 2024CLAMP-ViT: Contrastive Data-Free Learning for Adaptive Post-Training Quantization of ViTscited by 17
  4. 2023Auto-Prox: Training-Free Vision Transformer Architecture Search via Automatic Proxy Discoverycited by 13
  5. 2024NoRA: Nested Low-Rank Adaptation for Efficient Fine-Tuning Large Modelscited by 13
  6. 2024LPZero: Language Model Zero-cost Proxy Search from Zerocited by 7
  7. 2024Algorithm-Hardware Co-Design of Distribution-Aware Logarithmic-Posit Encodings for Efficient DNN Inferencecited by 6
  8. 2023Pruning Large Language Models via Accuracy Predictorcited by 5
  9. 2024ParZC: Parametric Zero-Cost Proxies for Efficient NAScited by 5
  10. 2024Bandwidth-Aware and Overlap-Weighted Compression for Communication-Efficient Federated Learningcited by 4
  11. 2025InfoQ: Mixed-Precision Quantization via Global Information Flowcited by 4
  12. 2023TVT: Training-Free Vision Transformer Search on Tiny Datasetscited by 3
10 more
  1. 2025MixA-Q: Revisiting Activation Sparsity for Vision Transformers from a Mixed-Precision Quantization Perspectivecited by 2
  2. 2025Enhancing Large Multimodal Models with Adaptive Sparsity and KV Cache Compressioncited by 2
  3. 2023On-Chip Hardware-Aware Quantization for Mixed Precision Neural Networkscited by 1
  4. 2025W-PCA Based Gradient-Free Proxy for Efficient Search of Lightweight Language Modelscited by 1
  5. 2025Structure-Aware Automatic Channel Pruning by Searching with Graph Embeddingcited by 1
  6. 2025Where and How to Enhance: Discovering Bit-Width Contribution for Mixed Precision Quantizationcited by 1
  7. 2023SONIQ: System-Optimized Noise-Injected Ultra-Low-Precision Quantization with Full-Precision Paritycited by 0
  8. 2025Towards Training-free Automatic Proxy Discovery via Large Language Models for Mixed Precision Quantizationcited by 0
  9. 2026Scale When Needed: Adaptive Neuron-level Mixed Precision Quantization Aware Trainingcited by 0
  10. 2026RAMP: Robust Adaptive Mixed-Precision Quantization for Edge CPU Vision Modelscited by 0