# Dual Dynamic Inference: Enabling More Efficient, Adaptive and Controllable Deep Inference

arXiv: https://arxiv.org/abs/1907.04523 · HTML: https://mlsanity.com/paper/1907.04523/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2019.

## Built on by (most cited first, 30 of 31)

- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizations](https://mlsanity.com/paper/2202.07800/) (2022)
- [IA-RED2: Interpretability-Aware Redundancy Reduction for Vision Transformers](https://mlsanity.com/paper/2106.12620/) (2021)
- [Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges](https://mlsanity.com/paper/2103.04505/) (2021)
- [HW-NAS-Bench:Hardware-Aware Neural Architecture Search Benchmark](https://mlsanity.com/paper/2103.10584/) (2021)
- [Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions](https://mlsanity.com/paper/2106.05022/) (2021)
- [Improving Transformers with Probabilistic Attention Keys](https://mlsanity.com/paper/2110.08678/) (2021)
- [Auditing Membership Leakages of Multi-Exit Networks](https://mlsanity.com/paper/2208.11180/) (2022)
- [Green Edge AI: A Contemporary Survey](https://mlsanity.com/paper/2312.00333/) (2023)
- [Layer- and Timestep-Adaptive Differentiable Token Compression Ratios for Efficient Diffusion Transformers](https://mlsanity.com/paper/2412.16822/) (2024)
- [Continual Object Detection via Prototypical Task Correlation Guided Gating Mechanism](https://mlsanity.com/paper/2205.03055/) (2022)
- [AppealNet: An Efficient and Highly-Accurate Edge/Cloud Collaborative Architecture for DNN Inference](https://mlsanity.com/paper/2105.04104/) (2021)
- [MIA-Former: Efficient and Robust Vision Transformers via Multi-grained Input-Adaptation](https://mlsanity.com/paper/2112.11542/) (2021)
- [2-in-1 Accelerator: Enabling Random Precision Switch for Winning Both Adversarial Robustness and Efficiency](https://mlsanity.com/paper/2109.05223/) (2021)
- [A Survey on Dynamic Neural Networks: from Computer Vision to Multi-modal Sensor Fusion](https://mlsanity.com/paper/2501.07451/) (2025)
- [DynaShare: Task and Instance Conditioned Parameter Sharing for Multi-Task Learning](https://mlsanity.com/paper/2305.17305/) (2023)
- [On the Impact of Black-box Deployment Strategies for Edge AI on Latency and Model Performance](https://mlsanity.com/paper/2403.17154/) (2024)
- [InstantNet: Automated Generation and Deployment of Instantaneously Switchable-Precision Networks](https://mlsanity.com/paper/2104.10853/) (2021)
- [LDP: Learnable Dynamic Precision for Efficient Deep Neural Network Training and Inference](https://mlsanity.com/paper/2203.07713/) (2022)
- [Input Conditioned Layer Dropping in Speech Foundation Models](https://mlsanity.com/paper/2507.07954/) (2025)
- [Vision Transformer Computation and Resilience for Dynamic Inference](https://mlsanity.com/paper/2212.02687/) (2022)
- [Predicting Probabilities of Error to Combine Quantization and Early Exiting: QuEE](https://mlsanity.com/paper/2406.14404/) (2024)
- [Two is Better than One: Efficient Ensemble Defense for Robust and Compact Models](https://mlsanity.com/paper/2504.04747/) (2025)
- [SmartDeal: Re-Modeling Deep Network Weights for Efficient Inference and Training](https://mlsanity.com/paper/2101.01163/) (2021)
- [Graceful Degradation and Related Fields](https://mlsanity.com/paper/2106.11119/) (2021)
- [TIPS: Topologically Important Path Sampling for Anytime Neural Networks](https://mlsanity.com/paper/2305.08021/) (2023)
- [Model Adaptation for Time Constrained Embodied Control](https://mlsanity.com/paper/2406.11128/) (2024)
- [Leveraging Stochastic Depth Training for Adaptive Inference](https://mlsanity.com/paper/2505.17626/) (2025)
- [Rethinking Inference Placement for Deep Learning across Edge and Cloud Platforms: A Multi-Objective Optimization Perspective and Future Directions](https://mlsanity.com/paper/2510.22909/) (2025)
- [Channel-Adaptive Edge AI: Maximizing Inference Throughput by Adapting Computational Complexity to Channel States](https://mlsanity.com/paper/2603.03146/) (2026)

## Abstract (arXiv)

State-of-the-art convolutional neural networks (CNNs) yield record-breaking predictive performance, yet at the cost of high-energy-consumption inference, that prohibits their widely deployments in resource-constrained Internet of Things (IoT) applications. We propose a dual dynamic inference (DDI) framework that highlights the following aspects: 1) we integrate both input-dependent and resource-dependent dynamic inference mechanisms under a unified framework in order to fit the varying IoT resource requirements in practice. DDI is able to both constantly suppress unnecessary costs for easy samples, and to halt inference for all samples to meet hard resource constraints enforced; 2) we propose a flexible multi-grained learning to skip (MGL2S) approach for input-dependent inference which allows simultaneous layer-wise and channel-wise skipping; 3) we extend DDI to complex CNN backbones such as DenseNet and show that DDI can be applied towards optimizing any specific resource goals including inference latency or energy cost. Extensive experiments demonstrate the superior inference accuracy-resource trade-off achieved by DDI, as well as the flexibility to control such trade-offs compared to existing peer methods. Specifically, DDI can achieve up to 4 times computational savings with the same or even higher accuracy as compared to existing competitive baselines.