mlsanityData to 7 Oct 2026

Differentiable Dynamic Quantization with Mixed Precision and Adaptive Resolution

arXiv 2106.02295 ↗First posted 4 Jun 2021cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

Model quantization is challenging due to many tedious hyper-parameters such as precision (bitwidth), dynamic range (minimum and maximum discrete values) and stepsize (interval between discrete values). Unlike prior arts that carefully tune these values, we present a fully differentiable approach to learn all of them, named Differentiable Dynamic Quantization (DDQ), which has several benefits. (1) DDQ is able to quantize challenging lightweight architectures like MobileNets, where different layers prefer different quantization parameters. (2) DDQ is hardware-friendly and can be easily implemented using low-precision matrix-vector multiplication, making it capable in many hardware such as ARM. (3) Extensive experiments show that DDQ outperforms prior arts on many networks and benchmarks, especially when models are already efficient and compact. e.g., DDQ is the first approach that achieves lossless 4-bit quantization for MobileNetV2 on ImageNet.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 38 of its 46 references that are in the map. First posted on arXiv 4 Jun 2021.

Builds on

38 of 46 references.

References · 38

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  3. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  4. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  5. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  7. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  8. 2019Searching for MobileNetV3cited by 1,984
  9. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  10. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  11. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  12. 2019Learned Step Size Quantizationcited by 604
26 more
  1. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  2. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  3. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  4. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  5. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  6. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  7. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  8. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 234
  9. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 211
  10. 2016Trained Ternary Quantizationcited by 203
  11. 2020LSQ+: Improving low-bit quantization through learnable offsets and better initializationcited by 180
  12. 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 157
  13. 2017CondenseNet: An Efficient DenseNet using Learned Group Convolutionscited by 121
  14. 2016Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1cited by 102
  15. 2016Convolutional Neural Networks using Logarithmic Data Representationcited by 97
  16. 2019Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networkscited by 87
  17. 2020Rethinking Differentiable Search for Mixed-Precision Neural Networkscited by 79
  18. 2020PROFIT: A Novel Training Method for sub-4-bit MobileNet Modelscited by 50
  19. 2017WRPN: Wide Reduced-Precision Networkscited by 49
  20. 2020HMQ: Hardware Friendly Mixed Precision Quantization Block for CNNscited by 43
  21. 2018Discovering Low-Precision Networks Close to Full-Precision Networks for Efficient Embedded Inferencecited by 33
  22. 2018NICE: Noise Injection and Clamping Estimation for Neural Network Quantizationcited by 24
  23. 2018Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)cited by 19
  24. 2019Towards Efficient Training for Neural Network Quantizationcited by 16
  25. 2018Fast Adjustable Threshold For Uniform Neural Network Quantization (Winning solution of LPIRC-II)cited by 6
  26. 2018Efficient non-uniform quantizer for quantized neural network targeting reconfigurable hardwarecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 4; 2024: 3; 2025: 5; 2026: 1.

19 papers.

Sort:

By area:

  1. 2022AdaBin: Improving Binary Neural Networks with Adaptive Binary Setscited by 38
  2. 2022SDQ: Stochastic Differentiable Quantization with Mixed Precisioncited by 27
  3. 2023Data Quality-aware Mixed-precision Quantization via Hybrid Reinforcement Learningcited by 17
  4. 2023Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devicescited by 11
  5. 2024Channel-Wise Mixed-Precision Quantization for Large Language Modelscited by 11
  6. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  7. 2023Real-time Controllable Denoising for Image and Videocited by 9
  8. 2024BiPer: Binary Neural Networks using a Periodic Functioncited by 9
  9. 2023MetaMix: Meta-state Precision Searcher for Mixed-precision Activation Quantizationcited by 7
  10. 2022AMED: Automatic Mixed-Precision Quantization for Edge Devicescited by 3
  11. 2022AskewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networkscited by 2
  12. 2022Exploiting the Partly Scratch-off Lottery Ticket for Quantization-Aware Trainingcited by 2
7 more
  1. 2025Towards Efficient and Accurate Spiking Neural Networks via Adaptive Bit Allocationcited by 2
  2. 202510K is Enough: An Ultra-Lightweight Binarized Network for Infrared Small-Target Detectioncited by 1
  3. 2025Where and How to Enhance: Discovering Bit-Width Contribution for Mixed Precision Quantizationcited by 1
  4. 2022Convolutional Neural Networks Quantization with Attentioncited by 0
  5. 2024DNN Memory Footprint Reduction via Post-Training Intra-Layer Multi-Precision Quantizationcited by 0
  6. 2025Towards Training-free Automatic Proxy Discovery via Large Language Models for Mixed Precision Quantizationcited by 0
  7. 2026Algorithm-Architecture Co-Design for Efficient VLA Inference via Speculative Inference and Verificationcited by 0