mlsanityData to 7 Oct 2026

Iterative Activation-based Structured Pruning

arXiv 2201.09881 ↗First posted 22 Jan 2022cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Deploying complex deep learning models on edge devices is challenging because they have substantial compute and memory resource requirements, whereas edge devices' resource budget is limited. To solve this problem, extensive pruning techniques have been proposed for compressing networks. Recent advances based on the Lottery Ticket Hypothesis (LTH) show that iterative model pruning tends to produce smaller and more accurate models. However, LTH research focuses on unstructured pruning, which is hardware-inefficient and difficult to accelerate on hardware platforms. In this paper, we investigate iterative pruning in the context of structured pruning because structurally pruned models map well on commodity hardware. We find that directly applying a structured weight-based pruning technique iteratively, called iterative L1-norm based pruning (ILP), does not produce accurate pruned models. To solve this problem, we propose two activation-based pruning methods, Iterative Activation-based Pruning (IAP) and Adaptive Iterative Activation-based Pruning (AIAP). We observe that, with only 1% accuracy loss, IAP and AIAP achieve 7.75X and 15.88X compression on LeNet-5, and 1.25X and 1.71X compression on ResNet-50, whereas ILP achieves 4.77X and 1.13X, respectively.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 19 of its 31 references that are in the map. First posted on arXiv 22 Jan 2022.

Builds on

19 of 31 references.

References · 19

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  3. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  4. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  5. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  6. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  7. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  8. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  9. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  10. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  11. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
  12. 2018Slimmable Neural Networkscited by 301
7 more
  1. 2018Model compression via distillation and quantizationcited by 287
  2. 2020Comparing Rewinding and Fine-tuning in Neural Network Pruningcited by 238
  3. 2019Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computingcited by 150
  4. 2015Data-free parameter pruning for Deep Neural Networkscited by 112
  5. 2018Faster gaze prediction with dense networks and Fisher pruningcited by 92
  6. 2019Fast Sparse ConvNetscited by 86
  7. 2018Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergycited by 50

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Elastic ViTs from Pretrained Models without Retrainingcited by 1
  2. 2026FedASAP: Activation Statistics-driven Structured Adaptive Pruning for Efficient Personalized Federated Learning for Lesion Segmentation on brain MRIcited by 0