mlsanityData to 7 Oct 2026

Rethinking Differentiable Search for Mixed-Precision Neural Networks

arXiv 2004.05795 ↗First posted 13 Apr 2020cs.LG cs.CV cs.NEcited by 79

Not on the map. Why?

Abstract (arXiv)

Low-precision networks, with weights and activations quantized to low bit-width, are widely used to accelerate inference on edge devices. However, current solutions are uniform, using identical bit-width for all filters. This fails to account for the different sensitivities of different filters and is suboptimal. Mixed-precision networks address this problem, by tuning the bit-width to individual filter requirements. In this work, the problem of optimal mixed-precision network search (MPS) is considered. To circumvent its difficulties of discrete search space and combinatorial optimization, a new differentiable search architecture is proposed, with several novel contributions to advance the efficiency by leveraging the unique properties of the MPS problem. The resulting Efficient differentiable MIxed-Precision network Search (EdMIPS) method is effective at finding the optimal bit allocation for multiple popular networks, and can search a large model, e.g. Inception-V3, directly on ImageNet without proxy task in a reasonable amount of time. The learned mixed-precision networks significantly outperform their uniform counterparts.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 15; 2023: 20; 2024: 12; 2025: 9; 2026: 7.

79 papers.

Sort:

By area:

  1. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scalingcited by 62
  2. 2022Mixed-Precision Neural Network Quantization via Learned Layer-wise Importancecited by 53
  3. 2022ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantizationcited by 42
  4. 2021Dynamic Network Quantization for Efficient Video Inferencecited by 36
  5. 2022Q-ViT: Fully Differentiable Quantization for Vision Transformercited by 36
  6. 2022SDQ: Stochastic Differentiable Quantization with Mixed Precisioncited by 27
  7. 2022CADyQ: Content-Aware Dynamic Quantization for Image Super-Resolutioncited by 26
  8. 2021Sharpness-aware Quantization for Deep Neural Networkscited by 25
  9. 2024Instance-Aware Group Quantization for Vision Transformerscited by 23
  10. 2022Collaborative Multi-Teacher Knowledge Distillation for Learning Low Bit-width Deep Neural Networkscited by 22
  11. 2023EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantizationcited by 22
  12. 2021Bit-Mixer: Mixed-precision networks with runtime bit-width selectioncited by 20
38 more
  1. 2021Differentiable Dynamic Quantization with Mixed Precision and Adaptive Resolutioncited by 19
  2. 2022Wavelet Feature Maps Compression for Image-to-Image CNNscited by 16
  3. 2022Q-PPG: Energy-Efficient PPG-based Heart Rate Monitoring on Wearable Devicescited by 14
  4. 2024Retraining-free Model Quantization via One-Shot Weight-Coupling Learningcited by 13
  5. 2021Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Acceleratorscited by 12
  6. 2021Sparse PointPillars: Maintaining and Exploiting Input Sparsity to Improve Runtime on Embedded Systemscited by 11
  7. 2021Generalizable Mixed-Precision Quantization via Attribution Rank Preservationcited by 11
  8. 2023Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devicescited by 11
  9. 2024BiDM: Pushing the Limit of Quantization for Diffusion Modelscited by 11
  10. 2023EQ-Net: Elastic Quantization Neural Networkscited by 10
  11. 2024MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformercited by 10
  12. 2022Channel-wise Mixed-precision Assignment for DNN Inference on Constrained Edge Nodescited by 9
  13. 2023PLiNIO: A User-Friendly Library of Gradient-based Methods for Complexity-aware DNN Optimizationcited by 9
  14. 2024TMPQ-DM: Joint Timestep Reduction and Quantization Precision Selection for Efficient Diffusion Modelscited by 9
  15. 2024RefQSR: Reference-based Quantization for Image Super-Resolution Networkscited by 8
  16. 2022Mixed-Precision Neural Networks: A Surveycited by 7
  17. 2023MetaMix: Meta-state Precision Searcher for Mixed-precision Activation Quantizationcited by 7
  18. 2021Single-path Bit Sharing for Automatic Loss-aware Model Compressioncited by 6
  19. 2021Effective and Fast: A Novel Sequential Single Path Search for Mixed-Precision Quantizationcited by 6
  20. 2022Human Activity Recognition on Microcontrollers with Quantized and Adaptive Deep Neural Networkscited by 6
  21. 2023SEAM: Searching Transferable Mixed-Precision Quantization Policy through Large Margin Regularizationcited by 6
  22. 2024Joint Pruning and Channel-wise Mixed-Precision Quantization for Efficient Deep Neural Networkscited by 6
  23. 2024Content-Aware Radiance Fields: Aligning Model Complexity with Scene Intricacy Through Learned Bitwidth Quantizationcited by 6
  24. 2021Improved Techniques for Quantizing Deep Networks with Adaptive Bit-Widthscited by 5
  25. 2021Mixed Precision Low-bit Quantization of Neural Network Language Models for Speech Recognitioncited by 5
  26. 2023FLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization Searchcited by 5
  27. 2021MWQ: Multiscale Wavelet Quantized Neural Networkscited by 4
  28. 2022QuantNAS for super resolution: searching for efficient quantization-friendly architectures against quantization noisecited by 4
  29. 2023Overcoming Distribution Mismatch in Quantizing Image Super-Resolution Networkscited by 4
  30. 2025Universality of Layer-Level Entropy-Weighted Quantization Beyond Model Architecture and Sizecited by 4
  31. 2025InfoQ: Mixed-Precision Quantization via Global Information Flowcited by 4
  32. 2025Mixed-Precision Quantization for Language Models: Techniques and Prospectscited by 4
  33. 2021One Model for All Quantization: A Quantized Network Supporting Hot-Swap Bit-Width Adjustmentcited by 3
  34. 2023Precision-aware Latency and Energy Balancing on Multi-Accelerator Platforms for DNN Inferencecited by 3
  35. 2023Free Bits: Latency Optimization of Mixed-Precision Quantized Neural Networks on the Edgecited by 3
  36. 2023Sensitivity-Aware Mixed-Precision Quantization and Width Optimization of Deep Neural Networks Through Cluster-Based Tree-Structured Parzen Estimationcited by 3
  37. 2025Learning from Loss Landscape: Generalizable Mixed-Precision Quantization via Adaptive Sharpness-Aware Gradient Aligningcited by 3
  38. 2025Efficient Mixed Precision Quantization in Graph Neural Networkscited by 3

50 most cited of 79 All 79 (JSON)