# DAIS: Automatic Channel Pruning via Differentiable Annealing Indicator Search

arXiv: https://arxiv.org/abs/2011.02166 · HTML: https://mlsanity.com/paper/2011.02166/

## Key facts

- Cited by 21 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 4 Nov 2020.

## Built on by (most cited first, 21 of 21)

- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization](https://mlsanity.com/paper/2206.01198/) (2022)
- [SGLP: A Similarity Guided Fast Layer Partition Pruning for Compressing Large Deep Models](https://mlsanity.com/paper/2410.14720/) (2024)
- [Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution](https://mlsanity.com/paper/2207.12577/) (2022)
- [CP3: Channel Pruning Plug-in for Point-based Networks](https://mlsanity.com/paper/2303.13097/) (2023)
- [An Effective Information Theoretic Framework for Channel Pruning](https://mlsanity.com/paper/2408.16772/) (2024)
- [CATRO: Channel Pruning via Class-Aware Trace Ratio Optimization](https://mlsanity.com/paper/2110.10921/) (2021)
- [Effective Layer Pruning Through Similarity Metric Perspective](https://mlsanity.com/paper/2405.17081/) (2024)
- [All-in-One: A Highly Representative DNN Pruning Framework for Edge Devices with Dynamic Power Management](https://mlsanity.com/paper/2212.05122/) (2022)
- [DASS: Differentiable Architecture Search for Sparse neural networks](https://mlsanity.com/paper/2207.06968/) (2022)
- [Exploring Gradient Flow Based Saliency for DNN Model Compression](https://mlsanity.com/paper/2110.12477/) (2021)
- [Latency Adjustable Transformer Encoder for Language Understanding](https://mlsanity.com/paper/2201.03327/) (2022)
- [Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Survey](https://mlsanity.com/paper/2208.10498/) (2022)
- [Adaptive Pruning of Deep Neural Networks for Resource-Aware Embedded Intrusion Detection on the Edge](https://mlsanity.com/paper/2505.14592/) (2025)
- [Real-Time Portrait Stylization on the Edge](https://mlsanity.com/paper/2206.01244/) (2022)
- [A Generalization of Continuous Relaxation in Structured Pruning](https://mlsanity.com/paper/2308.14605/) (2023)
- [A Greedy Hierarchical Approach to Whole-Network Filter-Pruning in CNNs](https://mlsanity.com/paper/2409.03777/) (2024)
- [Layer Pruning with Consensus: A Triple-Win Solution](https://mlsanity.com/paper/2411.14345/) (2024)
- [Pruning Everything, Everywhere, All at Once](https://mlsanity.com/paper/2506.04513/) (2025)
- [Explore and Establish Synergistic Effects Between Weight Pruning and Coreset Selection in Neural Network Training](https://mlsanity.com/paper/2511.09901/) (2025)
- [Zero-Inflated Gaussian Distributions Enable Parameter-Space Sparsity in Estimation-of-Distribution Algorithms](https://mlsanity.com/paper/2606.19369/) (2026)

## Abstract (arXiv)

The convolutional neural network has achieved great success in fulfilling computer vision tasks despite large computation overhead against efficient deployment. Structured (channel) pruning is usually applied to reduce the model redundancy while preserving the network structure, such that the pruned network can be easily deployed in practice. However, existing structured pruning methods require hand-crafted rules which may lead to tremendous pruning space. In this paper, we introduce Differentiable Annealing Indicator Search (DAIS) that leverages the strength of neural architecture search in the channel pruning and automatically searches for the effective pruned model with given constraints on computation overhead. Specifically, DAIS relaxes the binarized channel indicators to be continuous and then jointly learns both indicators and model parameters via bi-level optimization. To bridge the non-negligible discrepancy between the continuous model and the target binarized model, DAIS proposes an annealing-based procedure to steer the indicator convergence towards binarized states. Moreover, DAIS designs various regularizations based on a priori structural knowledge to control the pruning sparsity and to improve model performance. Experimental results show that DAIS outperforms state-of-the-art pruning methods on CIFAR-10, CIFAR-100, and ImageNet.