mlsanityData to 7 Oct 2026

PROFIT: A Novel Training Method for sub-4-bit MobileNet Models

arXiv 2008.04693 ↗First posted 11 Aug 2020cs.CV cs.PFcited by 50

Not on the map. Why?

Abstract (arXiv)

4-bit and lower precision mobile models are required due to the ever-increasing demand for better energy efficiency in mobile devices. In this work, we report that the activation instability induced by weight quantization (AIWQ) is the key obstacle to sub-4-bit quantization of mobile networks. To alleviate the AIWQ problem, we propose a novel training method called PROgressive-Freezing Iterative Training (PROFIT), which attempts to freeze layers whose weights are affected by the instability problem stronger than the other layers. We also propose a differentiable and unified quantization method (DuQ) and a negative padding idea to support asymmetric activation functions such as h-swish. We evaluate the proposed methods by quantizing MobileNet-v1, v2, and v3 on ImageNet and report that 4-bit quantization offers comparable (within 1.48 % top-1 accuracy) accuracy to full precision baseline. In the ablation study of the 3-bit quantization of MobileNet-v3, our proposed method outperforms the state-of-the-art method by a large margin, 12.86 % of top-1 accuracy.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 14; 2023: 10; 2024: 10; 2025: 6; 2026: 1.

50 papers.

Sort:

By area:

  1. 2022Overcoming Oscillations in Quantization-Aware Trainingcited by 116
  2. 2021Network Quantization with Element-wise Gradient Scalingcited by 82
  3. 2023Temporal Dynamic Quantization for Diffusion Modelscited by 81
  4. 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 78
  5. 2022Mixed-Precision Neural Network Quantization via Learned Layer-wise Importancecited by 53
  6. 2022NIPQ: Noise proxy-based Integrated Pseudo-Quantizationcited by 35
  7. 2021DKM: Differentiable K-Means Clustering Layer for Neural Network Compressioncited by 31
  8. 2022Optimal Clipping and Magnitude-aware Differentiation for Improved Quantization-aware Trainingcited by 26
  9. 2021Sharpness-aware Quantization for Deep Neural Networkscited by 25
  10. 2024Instance-Aware Group Quantization for Vision Transformerscited by 23
  11. 2021CPT: Efficient Deep Neural Network Training via Cyclic Precisioncited by 19
  12. 2021Differentiable Dynamic Quantization with Mixed Precision and Adaptive Resolutioncited by 19
38 more
  1. 2021Distance-aware Quantizationcited by 19
  2. 2024ESPACE: Dimensionality Reduction of Activations for Model Compressioncited by 15
  3. 2022Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devicescited by 12
  4. 2023PDP: Parameter-free Differentiable Pruning is All You Needcited by 12
  5. 2023MetaMix: Meta-state Precision Searcher for Mixed-precision Activation Quantizationcited by 7
  6. 2023SEAM: Searching Transferable Mixed-Precision Quantization Policy through Large Margin Regularizationcited by 6
  7. 2024Improving Quantization-aware Training of Low-Precision Network via Block Replacement on Full-Precision Counterpartcited by 6
  8. 20212-in-1 Accelerator: Enabling Random Precision Switch for Winning Both Adversarial Robustness and Efficiencycited by 4
  9. 2022Edge Inference with Fully Differentiable Quantized Mixed Precision Neural Networkscited by 4
  10. 2024Edge AI-Enabled Chicken Health Detection Based on Enhanced FCOS-Lite and Knowledge Distillationcited by 4
  11. 2025LOTION: Smoothing the Optimization Landscape for Quantized Trainingcited by 4
  12. 2022AMED: Automatic Mixed-Precision Quantization for Edge Devicescited by 3
  13. 2022Symmetry Regularization and Saturating Nonlinearity for Robust Quantizationcited by 3
  14. 2023Mixed Precision Post Training Quantization of Neural Networks with Sensitivity Guided Searchcited by 3
  15. 2024PikeLPN: Mitigating Overlooked Inefficiencies of Low-Precision Neural Networkscited by 3
  16. 2021Training Multi-bit Quantized and Binarized Networks with A Learnable Symmetric Quantizercited by 2
  17. 2022LG-LSQ: Learned Gradient Linear Symmetric Quantizationcited by 2
  18. 2022FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs with Dynamic Fixed-Point Representationcited by 2
  19. 2022Online Hybrid Lightweight Representations Learning: Its Application to Visual Trackingcited by 2
  20. 2022BiTAT: Neural Network Binarization with Task-dependent Aggregated Transformationcited by 2
  21. 2023Augmenting Hessians with Inter-Layer Dependencies for Mixed-Precision Post-Training Quantizationcited by 2
  22. 2024HLQ: Fast and Efficient Backpropagation via Hadamard Low-rank Quantizationcited by 2
  23. 2023The Hardware Impact of Quantization and Pruning for Weights in Spiking Neural Networkscited by 1
  24. 2023On-Chip Hardware-Aware Quantization for Mixed Precision Neural Networkscited by 1
  25. 2024Better Schedules for Low Precision Training of Deep Neural Networkscited by 1
  26. 2024Scheduling Weight Transitions for Quantization-Aware Trainingcited by 1
  27. 2024Torch2Chip: An End-to-end Customizable Deep Neural Network Compression and Deployment Toolkit for Prototype Hardware Accelerator Designcited by 1
  28. 2025Dedicated Inference Engine and Binary-Weight Neural Networks for Lightweight Instance Segmentationcited by 1
  29. 2025Coded Deep Learning: Framework and Algorithmcited by 1
  30. 2025PROM: Prioritize Reduction of Multiplications Over Lower Bit-Widths for Efficient CNNscited by 1
  31. 2025Where and How to Enhance: Discovering Bit-Width Contribution for Mixed Precision Quantizationcited by 1
  32. 2022A Closer Look at Hardware-Friendly Weight Quantizationcited by 0
  33. 2022Deep learning model compression using network sensitivity and gradientscited by 0
  34. 2023Fully Quantized Always-on Face Detector Considering Mobile Image Sensorscited by 0
  35. 2023IDKM: Memory Efficient Neural Network Quantization via Implicit, Differentiable k-Meanscited by 0
  36. 2024HEANA: A Hybrid Time-Amplitude Analog Optical Accelerator with Flexible Dataflows for Energy-Efficient CNN Inferencecited by 0
  37. 2025CompressNAS : A Fast and Efficient Technique for Model Compression using Decompositioncited by 0
  38. 2026QATMA: Quantization-Aware Training with Multimodal Alignment for Open-Vocabulary Object Detectioncited by 0