# DVM: A Bytecode Virtual Machine Approach for Dynamic Tensor Computation

arXiv: https://arxiv.org/abs/2603.24239 · HTML: https://mlsanity.com/paper/2603.24239/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 21 references that are in the map.
- First posted on arXiv 25 Mar 2026.

## Builds on (most cited first, 5 of 5)

- [Nimble: Efficiently Compiling Dynamic Neural Networks for Model Inference](https://mlsanity.com/paper/2006.03031/) (2020)
- [LazyTensor: combining eager execution with domain-specific compilers](https://mlsanity.com/paper/2102.13267/) (2021)
- [DISC: A Dynamic Shape Compiler for Machine Learning Workloads](https://mlsanity.com/paper/2103.05288/) (2021)
- [DyCL: Dynamic Neural Network Compilation Via Program Rewriting and Graph Optimization](https://mlsanity.com/paper/2307.04963/) (2023)
- [Tempo: Compiled Dynamic Deep Learning with Symbolic Dependence Graphs](https://mlsanity.com/paper/2501.05408/) (2025)

## Abstract (arXiv)

Dynamism is common in AI computation, e.g., the dynamic tensor shapes and the dynamic control flows in models. Due to the long compilation time, existing runtime compilation damages the model efficiency, while the offline compilers either suffer from the long compilation time and device memory footprint to cover all the possible execution instances of a dynamic model, or sacrifice optimization opportunities for usability. In this paper, we rethink the feasibility of runtime compilation for dynamic models and identify that the key for it to work is to speed up the compilation or hide the compilation overhead. To do this, we propose a real-time compiler, DVM. In DVM, we design a runtime operator compiler based on a bytecode virtual machine to perform effective and efficient compilation for each dynamic operator instance given its input. Specifically, instead of compiling programs into machine code, we encode the operator program into bytecode on the CPU and decode the bytecode into virtual instructions for direct execution on the NPU. Based on the runtime operator compiler, we further propose an operator fuser, which performs symbol-deduction-based fusion on static graphs and runtime fusion on dynamic graphs. Both pattern- and stacking-based fusion are supported to increase fusion opportunities. Evaluation on operators, subgraphs, and models shows that, compared with TorchInductor, PyTorch-eager and MindSpore-graph-O0, we are up to 11.77× better in terms of the operator/model efficiency and up to 5 orders of magnitude faster in terms of the maximum compilation time.