# Speculative Automated Refactoring of Imperative Deep Learning Programs to Graph Execution

arXiv: https://arxiv.org/abs/2504.05424 · HTML: https://mlsanity.com/paper/2504.05424/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 118 references that are in the map.
- First posted on arXiv 7 Apr 2025.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems](https://mlsanity.com/paper/1512.01274/) (2015)
- [Taxonomy of Real Faults in Deep Learning Systems](https://mlsanity.com/paper/1910.11015/) (2019)
- [A Comprehensive Study on Deep Learning Bug Characteristics](https://mlsanity.com/paper/1906.01388/) (2019)
- [Software Engineering Challenges of Deep Learning](https://mlsanity.com/paper/1810.12034/) (2018)
- [Repairing Deep Neural Networks: Fix Patterns and Challenges](https://mlsanity.com/paper/2005.00972/) (2020)
- [TensorFlow Eager: A Multi-Stage, Python-Embedded DSL for Machine Learning](https://mlsanity.com/paper/1903.01855/) (2019)
- [An Empirical Study on Deployment Faults of Deep Learning Based Mobile Applications](https://mlsanity.com/paper/2101.04930/) (2021)
- [A Toolkit for Generating Code Knowledge Graphs](https://mlsanity.com/paper/2002.09440/) (2020)
- [Decomposing Convolutional Neural Networks into Reusable and Replaceable Modules](https://mlsanity.com/paper/2110.07720/) (2021)
- [What Do Developers Ask About ML Libraries? A Large-scale Study Using Stack Overflow](https://mlsanity.com/paper/1906.11940/) (2019)
- [Understanding Performance Problems in Deep Learning Systems](https://mlsanity.com/paper/2112.01771/) (2021)
- [LazyTensor: combining eager execution with domain-specific compilers](https://mlsanity.com/paper/2102.13267/) (2021)
- [Design Smells in Deep Learning Programs: An Empirical Study](https://mlsanity.com/paper/2107.02279/) (2021)
- [The Product Beyond the Model -- An Empirical Study of Repositories of Open-Source ML Products](https://mlsanity.com/paper/2308.04328/) (2023)
- [AutoGraph: Imperative-style Coding with Graph-based Performance](https://mlsanity.com/paper/1810.08061/) (2018)
- [Scalpel: The Python Static Analysis Framework](https://mlsanity.com/paper/2202.11840/) (2022)
- [SOAR: A Synthesis Approach for Data Science API Refactoring](https://mlsanity.com/paper/2102.06726/) (2021)
- [Reusing Deep Neural Network Models through Model Re-engineering](https://mlsanity.com/paper/2304.00245/) (2023)
- [Ariadne: Analysis for Machine Learning Program](https://mlsanity.com/paper/1805.04058/) (2018)
- [From Monolith to Microservices: A Classification of Refactoring Approaches](https://mlsanity.com/paper/1807.10059/) (2018)
- [Challenges in Migrating Imperative Deep Learning Programs to Graph Execution: An Empirical Study](https://mlsanity.com/paper/2201.09953/) (2022)
- [Static Analysis for AWS Best Practices in Python Code](https://mlsanity.com/paper/2205.04432/) (2022)
- [Terra: Imperative-Symbolic Co-Execution of Imperative Deep Learning Programs](https://mlsanity.com/paper/2201.09210/) (2022)
- [Towards Safe Automated Refactoring of Imperative Deep Learning Programs to Graph Execution](https://mlsanity.com/paper/2308.11785/) (2023)
- [PoTo: A Hybrid Andersen's Points-to Analysis for Python](https://mlsanity.com/paper/2409.03918/) (2024)

## Abstract (arXiv)

Efficiency is essential to support ever-growing datasets, especially for Deep Learning (DL) systems. DL frameworks have traditionally embraced deferred execution-style DL code -- supporting symbolic, graph-based Deep Neural Network (DNN) computation. While scalable, such development is error-prone, non-intuitive, and difficult to debug. Consequently, more natural, imperative DL frameworks encouraging eager execution have emerged but at the expense of run-time performance. Though hybrid approaches aim for the "best of both worlds," using them effectively requires subtle considerations. Our key insight is that, while DL programs typically execute sequentially, hybridizing imperative DL code resembles parallelizing sequential code in traditional systems. Inspired by this, we present an automated refactoring approach that assists developers in determining which otherwise eagerly-executed imperative DL functions could be effectively and efficiently executed as graphs. The approach features novel static imperative tensor and side-effect analyses for Python. Due to its inherent dynamism, analyzing Python may be unsound; however, the conservative approach leverages a speculative (keyword-based) analysis for resolving difficult cases that informs developers of any assumptions made. The approach is: (i) implemented as a plug-in to the PyDev Eclipse IDE that integrates the WALA Ariadne analysis framework and (ii) evaluated on nineteen DL projects consisting of 132 KLOC. The results show that 326 of 766 candidate functions (42.56%) were refactorable, and an average relative speedup of 2.16x on performance tests was observed with negligible differences in model accuracy. The results indicate that the approach is useful in optimizing imperative DL code to its full potential.