mlsanityData to 14 Oct 2026

Exploiting the Partly Scratch-off Lottery Ticket for Quantization-Aware Training

arXiv 2211.08544 ↗First posted 12 Nov 2022cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Quantization-aware training (QAT) receives extensive popularity as it well retains the performance of quantized networks. In QAT, the contemporary experience is that all quantized weights are updated for an entire training process. In this paper, this experience is challenged based on an interesting phenomenon we observed. Specifically, a large portion of quantized weights reaches the optimal quantization level after a few training epochs, which we refer to as the partly scratch-off lottery ticket. This straightforward-yet-valuable observation naturally inspires us to zero out gradient calculations of these weights in the remaining training period to avoid meaningless updating. To effectively find the ticket, we develop a heuristic method, dubbed lottery ticket scratcher (LTS), which freezes a weight once the distance between the full-precision one and its quantization level is smaller than a controllable threshold. Surprisingly, the proposed LTS typically eliminates 50%-70% weight updating and 25%-35% FLOPs of the backward pass, while still resulting on par with or even better performance than the compared baseline. For example, compared with the baseline, LTS improves 2-bit MobileNetV2 by 5.05%, eliminating 46% weight updating and 23% FLOPs of the backward pass. Code is at url{https://github.com/zysxmu/LTS}.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 56 of its 69 references that are in the map. First posted on arXiv 12 Nov 2022.

Builds on

56 of 69 references.

References · 56

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  7. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  9. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  10. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  11. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  12. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
38 more
  1. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
  2. 2019Learned Step Size Quantizationcited by 628
  3. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 521
  4. 2019Rigging the Lottery: Making All Tickets Winnerscited by 495
  5. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
  6. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
  7. 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 292
  8. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 238
  9. 2019What's Hidden in a Randomly Weighted Neural Network?cited by 237
  10. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 217
  11. 2019Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcited by 207
  12. 2019Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterizationcited by 189
  13. 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 160
  14. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 148
  15. 2018Scalable Methods for 8-bit Training of Neural Networkscited by 142
  16. 2017Deep Rewiring: Training very sparse deep networkscited by 139
  17. 2022Overcoming Oscillations in Quantization-Aware Trainingcited by 119
  18. 2021Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Trainingcited by 110
  19. 2016Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1cited by 105
  20. 2021Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Maskscited by 103
  21. 2021Network Quantization with Element-wise Gradient Scalingcited by 87
  22. 2021Sparse Training via Boosting Pruning Plasticity with Neuroregenerationcited by 84
  23. 2020Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficientcited by 81
  24. 2019Towards Unified INT8 Training for Convolutional Neural Networkcited by 79
  25. 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 79
  26. 2020Rotated Binary Neural Networkcited by 77
  27. 2022GradMax: Growing Neural Networks using Gradient Informationcited by 66
  28. 2018Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutescited by 63
  29. 2018Norm matters: efficient and accurate normalization schemes in deep networkscited by 62
  30. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimizationcited by 61
  31. 2020FracBits: Mixed Precision Quantization via Fractional Bit-Widthscited by 55
  32. 2020Sparse Weight Activation Trainingcited by 45
  33. 2021IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantizationcited by 43
  34. 2019Training Quantized Neural Networks with a Full-precision Auxiliary Modulecited by 35
  35. 2017meProp: Sparsified Back Propagation for Accelerated Deep Learning with Reduced Overfittingcited by 34
  36. 2020High-Capacity Expert Binary Networkscited by 34
  37. 2021Distribution Adaptive INT8 Quantization for Training CNNscited by 28
  38. 2017Extremely Large Minibatch SGD: Training ResNet-50 on ImageNet in 15 Minutescited by 22

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 0; 2025: 1.

2 papers.

  1. 2023I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantizationcited by 19
  2. 2025Regularization-based Framework for Quantization-, Fault- and Variability-Aware Trainingcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Improving Quantization-aware Training of Low-Precision Network via Block Replacement on Full-Precision Counterpart
  2. 2024Scheduling Weight Transitions for Quantization-Aware Training
  3. 2026A Target-Centric Survey of Quantization-Aware Training
  4. 2026Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training
  5. 2022QFT: Post-training quantization via fast joint finetuning of all degrees of freedom
  6. 2024Toward INT4 Fixed-Point Training via Exploring Quantization Error for Gradients
  7. 2024Gradient-Free Training of Quantized Neural Networks
  8. 2026StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths