mlsanityData to 7 Oct 2026

AdaBits: Neural Network Quantization with Adaptive Bit-Widths

arXiv 1912.09666 ↗First posted 20 Dec 2019cs.CV cs.LG cs.NEcited by 63

Not on the map. Why?

Abstract (arXiv)

Deep neural networks with adaptive configurations have gained increasing attention due to the instant and flexible deployment of these models on platforms with different resource budgets. In this paper, we investigate a novel option to achieve this goal by enabling adaptive bit-widths of weights and activations in the model. We first examine the benefits and challenges of training quantized model with adaptive bit-widths, and then experiment with several approaches including direct adaptation, progressive training and joint training. We discover that joint training is able to produce comparable performance on the adaptive model as individual models. We further propose a new technique named Switchable Clipping Level (S-CL) to further improve quantized models at the lowest bit-width. With our proposed techniques applied on a bunch of models including MobileNet-V1/V2 and ResNet-50, we demonstrate that bit-width of weights and activations is a new option for adaptively executable deep neural networks, offering a distinct opportunity for improved accuracy-efficiency trade-off as well as instant adaptation according to the platform constraints in real-world applications.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 8; 2023: 11; 2024: 10; 2025: 9; 2026: 3.

63 papers.

Sort:

By area:

  1. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  2. 2021Dynamic Neural Networks: A Surveycited by 405
  3. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 64
  4. 2021IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantizationcited by 43
  5. 2021Teachers Do More Than Teach: Compressing Image-to-Image Modelscited by 37
  6. 2021Dynamic Network Quantization for Efficient Video Inferencecited by 36
  7. 2021GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarizationcited by 29
  8. 2022F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantizationcited by 26
  9. 2022CADyQ: Content-Aware Dynamic Quantization for Image Super-Resolutioncited by 26
  10. 2021Sharpness-aware Quantization for Deep Neural Networkscited by 25
  11. 2024AdaRevD: Adaptive Patch Exiting Reversible Decoder Pushes the Limit of Image Deblurringcited by 25
  12. 2023EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantizationcited by 22
38 more
  1. 2021Bit-Mixer: Mixed-precision networks with runtime bit-width selectioncited by 20
  2. 2021BatchQuant: Quantized-for-all Architecture Search with Robust Quantizercited by 19
  3. 2023Data Quality-aware Mixed-precision Quantization via Hybrid Reinforcement Learningcited by 17
  4. 2022Adaptive Patch Exiting for Scalable Single Image Super-Resolutioncited by 16
  5. 2024Retraining-free Model Quantization via One-Shot Weight-Coupling Learningcited by 13
  6. 2021Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Acceleratorscited by 12
  7. 2024AmoebaLLM: Constructing Any-Shape Large Language Models for Efficient and Instant Deploymentcited by 12
  8. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  9. 2022Synergistic Self-supervised and Quantization Learningcited by 10
  10. 2022Bitwidth-Adaptive Quantization-Aware Neural Network Training: A Meta-Learning Approachcited by 9
  11. 2023GradMDM: Adversarial Attack on Dynamic Networkscited by 8
  12. 2024RefQSR: Reference-based Quantization for Image Super-Resolution Networkscited by 8
  13. 2023Dynamic Token-Pass Transformers for Semantic Segmentationcited by 7
  14. 2021Effective and Fast: A Novel Sequential Single Path Search for Mixed-Precision Quantizationcited by 6
  15. 2021A Generalized Zero-Shot Quantization of Deep Convolutional Neural Networks via Learned Weights Statisticscited by 6
  16. 2023MBQuant: A Novel Multi-Branch Topology Method for Arbitrary Bit-width Network Quantizationcited by 6
  17. 2024Content-Aware Radiance Fields: Aligning Model Complexity with Scene Intricacy Through Learned Bitwidth Quantizationcited by 6
  18. 2025Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applicationscited by 6
  19. 2021Improved Techniques for Quantizing Deep Networks with Adaptive Bit-Widthscited by 5
  20. 2022Robust Binary Models by Pruning Randomly-initialized Networkscited by 5
  21. 2021MWQ: Multiscale Wavelet Quantized Neural Networkscited by 4
  22. 20212-in-1 Accelerator: Enabling Random Precision Switch for Winning Both Adversarial Robustness and Efficiencycited by 4
  23. 2022Vertical Layering of Quantized Neural Networks for Heterogeneous Inferencecited by 4
  24. 2023Overcoming Distribution Mismatch in Quantizing Image Super-Resolution Networkscited by 4
  25. 2023Tango: rethinking quantization for graph neural network training on GPUscited by 4
  26. 2025NestQuant: Post-Training Integer-Nesting Quantization for On-Device DNNcited by 4
  27. 2025DP-LLM: Runtime Model Adaptation with Dynamic Layer-wise Precision Assignmentcited by 4
  28. 2021One Model for All Quantization: A Quantized Network Supporting Hot-Swap Bit-Width Adjustmentcited by 3
  29. 2021Training for temporal sparsity in deep neural networks, application in video processingcited by 2
  30. 2024LUTMUL: Exceed Conventional FPGA Roofline Limit by LUT-based Efficient Multiplication for Neural Network Inferencecited by 2
  31. 2025DQT: Dynamic Quantization Training via Dequantization-Free Nested Integer Arithmeticcited by 2
  32. 2025FraQAT: Quantization Aware Training with Fractional bitscited by 2
  33. 2021InstantNet: Automated Generation and Deployment of Instantaneously Switchable-Precision Networkscited by 1
  34. 2023MetaGrad: Adaptive Gradient Quantization with Hypernetworkscited by 1
  35. 2023Learning Accurate Performance Predictors for Ultrafast Automated Model Compressioncited by 1
  36. 2024In-Distribution Consistency Regularization Improves the Generalization of Quantization-Aware Trainingcited by 1
  37. 2024Predicting Probabilities of Error to Combine Quantization and Early Exiting: QuEEcited by 1
  38. 2026MixQuant: Adaptive Mixed-Precision Quantization for Large Language Modelscited by 1

50 most cited of 63 All 63 (JSON)