# TVM: An Automated End-to-End Optimizing Compiler for Deep Learning

arXiv: https://arxiv.org/abs/1802.04799 · HTML: https://mlsanity.com/paper/1802.04799/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2018.

## Built on by (most cited first, 30 of 62)

- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [I-BERT: Integer-only BERT Quantization](https://mlsanity.com/paper/2101.01321/) (2021)
- [AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks](https://mlsanity.com/paper/2106.12379/) (2021)
- [Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads](https://mlsanity.com/paper/2105.05720/) (2021)
- [MQBench: Towards Reproducible and Deployable Model Quantization Benchmark](https://mlsanity.com/paper/2111.03759/) (2021)
- [A TinyML Platform for On-Device Continual Learning with Quantized Latent Replays](https://mlsanity.com/paper/2110.10486/) (2021)
- [Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications](https://mlsanity.com/paper/2204.11786/) (2022)
- [How Far Are We From AGI: Are LLMs All We Need?](https://mlsanity.com/paper/2405.10313/) (2024)
- [Tensor Program Optimization with Probabilistic Programs](https://mlsanity.com/paper/2205.13603/) (2022)
- [CoSA: Scheduling by Constrained Optimization for Spatial Accelerators](https://mlsanity.com/paper/2105.01898/) (2021)
- [Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution](https://mlsanity.com/paper/2207.12577/) (2022)
- [NxMTransformer: Semi-Structured Sparsification for Natural Language Understanding via ADMM](https://mlsanity.com/paper/2110.15766/) (2021)
- [Do We Need Anisotropic Graph Neural Networks?](https://mlsanity.com/paper/2104.01481/) (2021)
- [Equivariant Filters for Efficient Tracking in 3D Imaging](https://mlsanity.com/paper/2103.10255/) (2021)
- [The CoRa Tensor Compiler: Compilation for Ragged Tensors with Minimal Padding](https://mlsanity.com/paper/2110.10221/) (2021)
- [Improving Retrieval-Augmented Large Language Models via Data Importance Learning](https://mlsanity.com/paper/2307.03027/) (2023)
- [Lite Pose: Efficient Architecture Design for 2D Human Pose Estimation](https://mlsanity.com/paper/2205.01271/) (2022)
- [TLP: A Deep Learning-based Cost Model for Tensor Program Tuning](https://mlsanity.com/paper/2211.03578/) (2022)
- [Automated Deep Learning: Neural Architecture Search Is Not the End](https://mlsanity.com/paper/2112.09245/) (2021)
- [Query Processing on Tensor Computation Runtimes](https://mlsanity.com/paper/2203.01877/) (2022)
- [Learning to Reverse DNNs from AI Programs Automatically](https://mlsanity.com/paper/2205.10364/) (2022)
- [Revisiting Neuron Coverage for DNN Testing: A Layer-Wise and Distribution-Aware Criterion](https://mlsanity.com/paper/2112.01955/) (2021)
- [UNIT: Unifying Tensorized Instruction Compilation](https://mlsanity.com/paper/2101.08458/) (2021)
- [GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsity](https://mlsanity.com/paper/2108.11033/) (2021)
- [Coverage-Guided Tensor Compiler Fuzzing with Joint IR-Pass Mutation](https://mlsanity.com/paper/2202.09947/) (2022)
- [MetaTune: Meta-Learning Based Cost Model for Fast and Efficient Auto-tuning Frameworks](https://mlsanity.com/paper/2102.04199/) (2021)
- [Arithmetic-Intensity-Guided Fault Tolerance for Neural Network Inference on GPUs](https://mlsanity.com/paper/2104.09455/) (2021)
- [Finding the Task-Optimal Low-Bit Sub-Distribution in Deep Neural Networks](https://mlsanity.com/paper/2112.15139/) (2021)
- [Operator Fusion in XLA: Analysis and Evaluation](https://mlsanity.com/paper/2301.13062/) (2023)
- [Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration](https://mlsanity.com/paper/2111.11581/) (2021)

## Abstract (arXiv)

There is an increasing need to bring machine learning to a wide diversity of hardware devices. Current frameworks rely on vendor-specific operator libraries and optimize for a narrow range of server-class GPUs. Deploying workloads to new platforms -- such as mobile phones, embedded devices, and accelerators (e.g., FPGAs, ASICs) -- requires significant manual effort. We propose TVM, a compiler that exposes graph-level and operator-level optimizations to provide performance portability to deep learning workloads across diverse hardware back-ends. TVM solves optimization challenges specific to deep learning, such as high-level operator fusion, mapping to arbitrary hardware primitives, and memory latency hiding. It also automates optimization of low-level programs to hardware characteristics by employing a novel, learning-based cost modeling method for rapid exploration of code optimizations. Experimental results show that TVM delivers performance across hardware back-ends that are competitive with state-of-the-art, hand-tuned libraries for low-power CPU, mobile GPU, and server-class GPUs. We also demonstrate TVM's ability to target new accelerator back-ends, such as the FPGA-based generic deep learning accelerator. The system is open sourced and in production use inside several major companies.