mlsanityData to 14 Oct 2026

COLT: Cyclic Overlapping Lottery Tickets for Faster Pruning of Convolutional Neural Networks

arXiv 2212.12770 ↗First posted 24 Dec 2022cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Pruning refers to the elimination of trivial weights from neural networks. The sub-networks within an overparameterized model produced after pruning are often called Lottery tickets. This research aims to generate winning lottery tickets from a set of lottery tickets that can achieve similar accuracy to the original unpruned network. We introduce a novel winning ticket called Cyclic Overlapping Lottery Ticket (COLT) by data splitting and cyclic retraining of the pruned network from scratch. We apply a cyclic pruning algorithm that keeps only the overlapping weights of different pruned models trained on different data segments. Our results demonstrate that COLT can achieve similar accuracies (obtained by the unpruned model) while maintaining high sparsities. We show that the accuracy of COLT is on par with the winning tickets of Lottery Ticket Hypothesis (LTH) and, at times, is better. Moreover, COLTs can be generated using fewer iterations than tickets generated by the popular Iterative Magnitude Pruning (IMP) method. In addition, we also notice COLTs generated on large datasets can be transferred to small ones without compromising performance, demonstrating its generalizing capability. We conduct all our experiments on Cifar-10, Cifar-100 & TinyImageNet datasets and report superior performance than the state-of-the-art methods.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 39 of its 59 references that are in the map. First posted on arXiv 24 Dec 2022.

Builds on

39 of 59 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  8. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  9. 2014How transferable are features in deep neural networks?cited by 1,225
  10. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  11. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  12. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
27 more
  1. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  2. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 659
  3. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 637
  4. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 603
  5. 2018Do Better ImageNet Models Transfer Better?cited by 565
  6. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 527
  7. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 479
  8. 2019The State of Sparsity in Deep Neural Networkscited by 476
  9. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 442
  10. 2018A Survey on Deep Transfer Learningcited by 419
  11. 2014In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learningcited by 342
  12. 2018Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layerscited by 340
  13. 2017Variational Dropout Sparsifies Deep Neural Networkscited by 265
  14. 2016Dynamic Network Surgery for Efficient DNNscited by 233
  15. 2019Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcited by 207
  16. 2019What Do Compressed Deep Neural Networks Forget?cited by 171
  17. 2019Quantization Networkscited by 153
  18. 2016Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruningcited by 134
  19. 2018On the Power of Over-parametrization in Neural Networks with Quadratic Activationcited by 102
  20. 2019One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizerscited by 102
  21. 2019Implicit Regularization for Optimal Sparse Recoverycited by 80
  22. 2019The Generalization-Stability Tradeoff In Neural Network Pruningcited by 54
  23. 2021Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?cited by 52
  24. 2022FairGRAPE: Fairness-aware GRAdient Pruning mEthod for Face Attribute Classificationcited by 30
  25. 2021Lottery Ticket Preserves Weight Correlation: Is It Desirable or Not?cited by 22
  26. 2021Anchor Pruning for Object Detectioncited by 5
  27. 2018Functionality-Oriented Convolutional Filter Pruningcited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 20243D Point Cloud Network Pruning: When Some Weights Do not Mattercited by 3
  2. 2025Z-Pruner: Post-Training Pruning of Large Language Models for Efficiency without Retrainingcited by 1
  3. 2025Teacher-Guided One-Shot Pruning via Context-Aware Knowledge Distillationcited by 0