# DyCL: Dynamic Neural Network Compilation Via Program Rewriting and Graph Optimization

arXiv: https://arxiv.org/abs/2307.04963 · HTML: https://mlsanity.com/paper/2307.04963/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 54 references that are in the map.
- First posted on arXiv 11 Jul 2023.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [SkipNet: Learning Dynamic Routing in Convolutional Networks](https://mlsanity.com/paper/1711.09485/) (2017)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [Ansor: Generating High-Performance Tensor Programs for Deep Learning](https://mlsanity.com/paper/2006.06762/) (2020)
- [Convolutional Networks with Adaptive Inference Graphs](https://mlsanity.com/paper/1711.11503/) (2017)
- [AdaFrame: Adaptive Frame Selection for Fast Video Recognition](https://mlsanity.com/paper/1811.12432/) (2018)
- [Dynamic Channel Pruning: Feature Boosting and Suppression](https://mlsanity.com/paper/1810.05331/) (2018)
- [Spatially Adaptive Computation Time for Residual Networks](https://mlsanity.com/paper/1612.02297/) (2016)
- [Channel Gating Neural Networks](https://mlsanity.com/paper/1805.12549/) (2018)
- [Dynamic Deep Neural Networks: Optimizing Accuracy-Efficiency Trade-offs by Selective Execution](https://mlsanity.com/paper/1701.00299/) (2017)
- [SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learning](https://mlsanity.com/paper/2207.04606/) (2022)
- [AutoFocus: Efficient Multi-Scale Inference](https://mlsanity.com/paper/1812.01600/) (2018)
- [The Deep Learning Compiler: A Comprehensive Survey](https://mlsanity.com/paper/2002.03794/) (2020)
- [Low-Rank Approximations for Conditional Feedforward Computation in Deep Neural Networks](https://mlsanity.com/paper/1312.4461/) (2013)
- [NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers](https://mlsanity.com/paper/2207.13066/) (2022)
- [Dynamic Zoom-in Network for Fast Object Detection in Large Images](https://mlsanity.com/paper/1711.05187/) (2017)
- [Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning](https://mlsanity.com/paper/1801.08058/) (2018)
- [IOS: Inter-Operator Scheduler for CNN Acceleration](https://mlsanity.com/paper/2011.01302/) (2020)
- [Nimble: Efficiently Compiling Dynamic Neural Networks for Model Inference](https://mlsanity.com/paper/2006.03031/) (2020)
- [DeepPerform: An Efficient Approach for Performance Testing of Resource-Constrained Neural Networks](https://mlsanity.com/paper/2210.05370/) (2022)
- [Cortex: A Compiler for Recursive Deep Learning Models](https://mlsanity.com/paper/2011.01383/) (2020)
- [LLMEffiChecker: Understanding and Testing Efficiency Degradation of Large Language Models](https://mlsanity.com/paper/2210.03696/) (2022)
- [Adaptive Classification for Prediction Under a Budget](https://mlsanity.com/paper/1705.10194/) (2017)
- [Learning to Reverse DNNs from AI Programs Automatically](https://mlsanity.com/paper/2205.10364/) (2022)

## Built on by (most cited first, 9 of 9)

- [RT-LM: Uncertainty-Aware Resource Management for Real-Time Inference of Language Models](https://mlsanity.com/paper/2309.06619/) (2023)
- [Your Compiler is Backdooring Your Model: Understanding and Exploiting Compilation Inconsistency Vulnerabilities in Deep Learning Compilers](https://mlsanity.com/paper/2509.11173/) (2025)
- [Yggdrasil: Bridging Dynamic Speculation and Static Runtime for Latency-Optimal Tree-Based LLM Decoding](https://mlsanity.com/paper/2512.23858/) (2025)
- [LeMix: Unified Scheduling for LLM Training and Inference on Multi-GPU Systems](https://mlsanity.com/paper/2507.21276/) (2025)
- [10Cache: Heterogeneous Resource-Aware Tensor Caching and Migration for LLM Training](https://mlsanity.com/paper/2511.14124/) (2025)
- [NGDB-Zoo: Towards Efficient and Scalable Neural Graph Databases Training](https://mlsanity.com/paper/2602.21597/) (2026)
- [Edge AI for Internet of Energy: Challenges and Perspectives](https://mlsanity.com/paper/2311.16851/) (2023)
- [Towards Migrating Neural Network Implementations](https://mlsanity.com/paper/2511.02610/) (2025)
- [DVM: A Bytecode Virtual Machine Approach for Dynamic Tensor Computation](https://mlsanity.com/paper/2603.24239/) (2026)

## Abstract (arXiv)

DL compiler's primary function is to translate DNN programs written in high-level DL frameworks such as PyTorch and TensorFlow into portable executables. These executables can then be flexibly executed by the deployed host programs. However, existing DL compilers rely on a tracing mechanism, which involves feeding a runtime input to a neural network program and tracing the program execution paths to generate the computational graph necessary for compilation. Unfortunately, this mechanism falls short when dealing with modern dynamic neural networks (DyNNs) that possess varying computational graphs depending on the inputs. Consequently, conventional DL compilers struggle to accurately compile DyNNs into executable code. To address this limitation, we propose tool, a general approach that enables any existing DL compiler to successfully compile DyNNs. tool tackles the dynamic nature of DyNNs by introducing a compilation mechanism that redistributes the control and data flow of the original DNN programs during the compilation process. Specifically, tool develops program analysis and program transformation techniques to convert a dynamic neural network into multiple sub-neural networks. Each sub-neural network is devoid of conditional statements and is compiled independently. Furthermore, tool synthesizes a host module that models the control flow of the DyNNs and facilitates the invocation of the sub-neural networks. Our evaluation demonstrates the effectiveness of tool, achieving a 100% success rate in compiling all dynamic neural networks. Moreover, the compiled executables generated by tool exhibit significantly improved performance, running between 1.12× and 20.21× faster than the original DyNNs executed on general-purpose DL frameworks.