mlsanityData to 14 Oct 2026

MetaMix: Meta-state Precision Searcher for Mixed-precision Activation Quantization

arXiv 2311.06798 ↗First posted 12 Nov 2023cs.LG cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Mixed-precision quantization of efficient networks often suffer from activation instability encountered in the exploration of bit selections. To address this problem, we propose a novel method called MetaMix which consists of bit selection and weight training phases. The bit selection phase iterates two steps, (1) the mixed-precision-aware weight update, and (2) the bit-search training with the fixed mixed-precision-aware weights, both of which combined reduce activation instability in mixed-precision quantization and contribute to fast and high-quality bit selection. The weight training phase exploits the weights and step sizes trained in the bit selection phase and fine-tunes them thereby offering fast training. Our experiments with efficient and hard-to-quantize networks, i.e., MobileNet v2 and v3, and ResNet-18 on ImageNet show that our proposed method pushes the boundary of mixed-precision quantization, in terms of accuracy vs. operations, by outperforming both mixed- and single-precision SOTA methods.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 35 of its 46 references that are in the map. First posted on arXiv 12 Nov 2023.

Builds on

35 of 46 references.

References · 35

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  4. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  5. 2019Searching for MobileNetV3cited by 1,994
  6. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  7. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  8. 2019Learned Step Size Quantizationcited by 628
  9. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  10. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  11. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  12. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 414
23 more
  1. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
  2. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 274
  3. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
  4. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 238
  5. 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 160
  6. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 148
  7. 2018Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Searchcited by 125
  8. 2022Overcoming Oscillations in Quantization-Aware Trainingcited by 119
  9. 2017Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networkscited by 109
  10. 2020Rethinking Differentiable Search for Mixed-Precision Neural Networkscited by 82
  11. 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 79
  12. 2020DSNAS: Direct Neural Architecture Search without Parameter Retrainingcited by 69
  13. 2020HAWQV3: Dyadic Neural Network Quantizationcited by 60
  14. 2020FracBits: Mixed Precision Quantization via Fractional Bit-Widthscited by 55
  15. 2020PROFIT: A Novel Training Method for sub-4-bit MobileNet Modelscited by 50
  16. 2020Differentiable Joint Pruning and Quantization for Hardware Efficiencycited by 46
  17. 2020HMQ: Hardware Friendly Mixed Precision Quantization Block for CNNscited by 43
  18. 2021Learnable Companding Quantization for Accurate Low-bit Neural Networkscited by 42
  19. 2022NIPQ: Noise proxy-based Integrated Pseudo-Quantizationcited by 35
  20. 2020Search What You Want: Barrier Panelty NAS for Mixed Precision Quantizationcited by 32
  21. 2021Differentiable Dynamic Quantization with Mixed Precision and Adaptive Resolutioncited by 20
  22. 2019Towards Efficient Training for Neural Network Quantizationcited by 16
  23. 2018ReLeQ: A Reinforcement Learning Approach for Deep Quantization of Neural Networkscited by 14

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 6.

7 papers.

  1. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  2. 2025Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applicationscited by 7
  3. 2024MetaAug: Meta-Data Augmentation for Post-Training Quantizationcited by 5
  4. 2025MixA-Q: Revisiting Activation Sparsity for Vision Transformers from a Mixed-Precision Quantization Perspectivecited by 3
  5. 2025Towards Efficient and Accurate Spiking Neural Networks via Adaptive Bit Allocationcited by 2
  6. 2025LampQ: Towards Accurate Layer-wise Mixed Precision Quantization for Vision Transformerscited by 2
  7. 2025Where and How to Enhance: Discovering Bit-Width Contribution for Mixed Precision Quantizationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2021Bit-Mixer: Mixed-precision networks with runtime bit-width selection
  2. 2021Effective and Fast: A Novel Sequential Single Path Search for Mixed-Precision Quantization
  3. 2023Free Bits: Latency Optimization of Mixed-Precision Quantized Neural Networks on the Edge
  4. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimization
  5. 2023FLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization Search
  6. 2023A Practical Mixed Precision Algorithm for Post-Training Quantization
  7. 2023QBitOpt: Fast and Accurate Bitwidth Reallocation during Training
  8. 2021Activation Density based Mixed-Precision Quantization for Energy Efficient Neural Networks