# S2DNAS:Transforming Static CNN Model for Dynamic Inference via Neural Architecture Search

arXiv: https://arxiv.org/abs/1911.07033 · HTML: https://mlsanity.com/paper/1911.07033/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2019.

## Built on by (most cited first, 14 of 14)

- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [MicroNet: Improving Image Recognition with Extremely Low FLOPs](https://mlsanity.com/paper/2108.05894/) (2021)
- [Controllable Dynamic Multi-Task Architectures](https://mlsanity.com/paper/2203.14949/) (2022)
- [Multi-Exit Semantic Segmentation Networks](https://mlsanity.com/paper/2106.03527/) (2021)
- [Efficient Deep Visual and Inertial Odometry with Adaptive Visual Modality Selection](https://mlsanity.com/paper/2205.06187/) (2022)
- [HADAS: Hardware-Aware Dynamic Neural Architecture Search for Edge Performance Scaling](https://mlsanity.com/paper/2212.03354/) (2022)
- [APG: Adaptive Parameter Generation Network for Click-Through Rate Prediction](https://mlsanity.com/paper/2203.16218/) (2022)
- [Dynamic Slimmable Denoising Network](https://mlsanity.com/paper/2110.08940/) (2021)
- [Map-and-Conquer: Energy-Efficient Mapping of Dynamic Neural Nets onto Heterogeneous MPSoCs](https://mlsanity.com/paper/2302.12926/) (2023)
- [SlowFormer: Universal Adversarial Patch for Attack on Compute and Energy Efficiency of Inference Efficient Vision Transformers](https://mlsanity.com/paper/2310.02544/) (2023)
- [Adaptive LiDAR Scanning: Harnessing Temporal Cues for Efficient 3D Object Detection via Multi-Modal Fusion](https://mlsanity.com/paper/2508.01562/) (2025)
- [ClassPruning: Speed Up Image Restoration Networks by Dynamic N:M Pruning](https://mlsanity.com/paper/2211.05488/) (2022)
- [TIPS: Topologically Important Path Sampling for Anytime Neural Networks](https://mlsanity.com/paper/2305.08021/) (2023)
- [HERCULES: Hardware-Efficient, Robust, Continual Learning Neural Architecture Search](https://mlsanity.com/paper/2605.04103/) (2026)

## Abstract (arXiv)

Recently, dynamic inference has emerged as a promising way to reduce the computational cost of deep convolutional neural network (CNN). In contrast to static methods (e.g. weight pruning), dynamic inference adaptively adjusts the inference process according to each input sample, which can considerably reduce the computational cost on "easy" samples while maintaining the overall model performance. In this paper, we introduce a general framework, S2DNAS, which can transform various static CNN models to support dynamic inference via neural architecture search. To this end, based on a given CNN model, we first generate a CNN architecture space in which each architecture is a multi-stage CNN generated from the given model using some predefined transformations. Then, we propose a reinforcement learning based approach to automatically search for the optimal CNN architecture in the generated space. At last, with the searched multi-stage network, we can perform dynamic inference by adaptively choosing a stage to evaluate for each sample. Unlike previous works that introduce irregular computations or complex controllers in the inference or re-design a CNN model from scratch, our method can generalize to most of the popular CNN architectures and the searched dynamic network can be directly deployed using existing deep learning frameworks in various hardware devices.