mlsanityData to 14 Oct 2026

Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization

arXiv 2206.01198 ↗First posted 2 Jun 2022cs.CVcited by 32

Not on the map. Why?

Abstract (arXiv)

Neural architecture search (NAS) and network pruning are widely studied efficient AI techniques, but not yet perfect. NAS performs exhaustive candidate architecture search, incurring tremendous search cost. Though (structured) pruning can simply shrink model dimension, it remains unclear how to decide the per-layer sparsity automatically and optimally. In this work, we revisit the problem of layer-width optimization and propose Pruning-as-Search (PaS), an end-to-end channel pruning method to search out desired sub-network automatically and efficiently. Specifically, we add a depth-wise binary convolution to learn pruning policies directly through gradient descent. By combining the structural reparameterization and PaS, we successfully searched out a new family of VGG-like and lightweight networks, which enable the flexibility of arbitrary width with respect to each layer instead of each stage. Experimental results show that our proposed architecture outperforms prior arts by around 1.0% top-1 accuracy under similar inference speed on ImageNet-1000 classification task. Furthermore, we demonstrate the effectiveness of our width search on complex tasks including instance segmentation and image translation. Code and models are released.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 27 of its 33 references that are in the map. First posted on arXiv 2 Jun 2022.

Builds on

27 of 33 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  3. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  4. 2017Squeeze-and-Excitation Networkscited by 4,789
  5. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  6. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  7. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  8. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  9. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  10. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  11. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 702
  12. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
15 more
  1. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 522
  2. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
  3. 2019YOLACT: Real-time Instance Segmentationcited by 443
  4. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 442
  5. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 378
  6. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 227
  7. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 221
  8. 2018ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Contentcited by 89
  9. 2020DMCP: Differentiable Markov Channel Pruning for Neural Networkscited by 69
  10. 2021Teachers Do More Than Teach: Compressing Image-to-Image Modelscited by 37
  11. 2021GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarizationcited by 29
  12. 2021BCNet: Searching for Network Width with Bilaterally Coupled Networkcited by 23
  13. 2020DAIS: Automatic Channel Pruning via Differentiable Annealing Indicator Searchcited by 21
  14. 2018StructADMM: A Systematic, High-Efficiency Framework of Structured Weight Pruning for DNNscited by 8
  15. 2018Systematic Weight Pruning of DNNs using Alternating Direction Method of Multiplierscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 14; 2025: 8; 2026: 3.

32 papers.

Sort:

By area:

  1. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  2. 2024Finding Transformer Circuits with Edge Pruningcited by 60
  3. 2024LazyDiT: Lazy Learning for the Acceleration of Diffusion Transformerscited by 37
  4. 2024Numerical Pruning for Efficient Autoregressive Modelscited by 25
  5. 2024Pruning Foundation Models for High Accuracy without Retrainingcited by 20
  6. 2023Text-Visual Prompting for Efficient 2D Temporal Video Groundingcited by 18
  7. 2024Squat: Quant Small Language Models on the Edgecited by 16
  8. 2024Search for Efficient Large Language Modelscited by 14
  9. 2024Fast and Memory-Efficient Video Diffusion Using Streamlined Inferencecited by 14
  10. 2024Toward Adaptive Large Language Models Structured Pruning via Hybrid-grained Weight Importance Assessmentcited by 13
  11. 2025Structured Agent Distillation for Large Language Modelcited by 13
  12. 2023Performance-aware Approximation of Global Channel Pruning for Multitask CNNscited by 9
20 more
  1. 2025RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptationcited by 9
  2. 2022Equivariance-aware Architectural Optimization of Neural Networkscited by 8
  3. 2025QuartDepth: Post-Training Quantization for Real-Time Depth Estimation on the Edgecited by 8
  4. 20247B Fully Open Source Moxin-LLM/VLM -- From Pretraining to GRPO-based Reinforcement Learning Enhancementcited by 5
  5. 2025Open-Source Multimodal Moxin Models with Moxin-VLM and Moxin-VLAcited by 5
  6. 2023UPSCALE: Unconstrained Channel Pruningcited by 4
  7. 2025On the creation of narrow AI: hierarchy and nonlocality of neural network skillscited by 4
  8. 2024Automatic Channel Pruning for Multi-Head Attentioncited by 3
  9. 2025Collaborative Compression for Large-Scale MoE Deployment on Edgecited by 3
  10. 2024Differentiable Model Scaling using Differentiable Topkcited by 2
  11. 2023End-to-End Neural Network Compression via fracℓ1ℓ2 Regularized Latency Surrogatescited by 1
  12. 2024Separate, Dynamic and Differentiable (SMART) Pruner for Block/Output Channel Pruning on Computer Vision Taskscited by 1
  13. 2025TSLA: A Task-Specific Learning Adaptation for Semantic Segmentation on Autonomous Vehicles Platformcited by 1
  14. 2023A Survey on Multi-Objective Neural Architecture Searchcited by 0
  15. 2024Designing Semi-Structured Pruning of Graph Convolutional Networks for Skeleton-based Recognitioncited by 0
  16. 2024Learning Coarse-to-Fine Pruning of Graph Convolutional Networks for Skeleton-based Recognitioncited by 0
  17. 2025Pruning Increases Orderedness in Recurrent Computationcited by 0
  18. 2026Automated Model Design using Gated Neuron Selection in Telecomcited by 0
  19. 2026Mechanistic Interpretability of Structure-Aware Numerical Reasoning in LLaMA 3.1 8Bcited by 0
  20. 2026From Patching to Pruning Visual Computation in Vision Language Modelscited by 0