# Ansor: Generating High-Performance Tensor Programs for Deep Learning

arXiv: https://arxiv.org/abs/2006.06762 · HTML: https://mlsanity.com/paper/2006.06762/

## Key facts

- Cited by 201 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2020.

## Built on by (most cited first, 30 of 201)

- [SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification](https://mlsanity.com/paper/2305.09781/) (2023)
- [OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantization](https://mlsanity.com/paper/2304.07493/) (2023)
- [Full Stack Optimization of Transformer Inference: a Survey](https://mlsanity.com/paper/2302.14017/) (2023)
- [Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Training](https://mlsanity.com/paper/2208.06102/) (2022)
- [ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization](https://mlsanity.com/paper/2208.14286/) (2022)
- [Astra: A Multi-Agent System for GPU Kernel Performance Optimization](https://mlsanity.com/paper/2509.07506/) (2025)
- [SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learning](https://mlsanity.com/paper/2207.04606/) (2022)
- [TileLang: A Composable Tiled Programming Model for AI Systems](https://mlsanity.com/paper/2504.17577/) (2025)
- [CompilerGym: Robust, Performant Compiler Optimization Environments for AI Research](https://mlsanity.com/paper/2109.08267/) (2021)
- [Mirage: A Multi-Level Superoptimizer for Tensor Programs](https://mlsanity.com/paper/2405.05751/) (2024)
- [TensorIR: An Abstraction for Automatic Tensorized Program Optimization](https://mlsanity.com/paper/2207.04296/) (2022)
- [MultiKernelBench: A Multi-Platform Benchmark for Kernel Generation](https://mlsanity.com/paper/2507.17773/) (2025)
- [Tensor Program Optimization with Probabilistic Programs](https://mlsanity.com/paper/2205.13603/) (2022)
- [NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers](https://mlsanity.com/paper/2207.13066/) (2022)
- [Hidet: Task-Mapping Programming Paradigm for Deep Learning Tensor Programs](https://mlsanity.com/paper/2210.09603/) (2022)
- [Flashlight: Enabling Innovation in Tools for Machine Learning](https://mlsanity.com/paper/2201.12465/) (2022)
- [PIT: Optimization of Dynamic Sparse Deep Learning Models via Permutation Invariant Transformation](https://mlsanity.com/paper/2301.10936/) (2023)
- [STARK: Strategic Team of Agents for Refining Kernels](https://mlsanity.com/paper/2510.16996/) (2025)
- [FlashInfer-Bench: Building the Virtuous Cycle for AI-driven LLM Systems](https://mlsanity.com/paper/2601.00227/) (2026)
- [K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model](https://mlsanity.com/paper/2602.19128/) (2026)
- [AccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimization](https://mlsanity.com/paper/2511.15915/) (2025)
- [Characterizing Mobile SoC for Accelerating Heterogeneous LLM Inference](https://mlsanity.com/paper/2501.14794/) (2025)
- [Bolt: Bridging the Gap between Auto-tuners and Hardware-native Performance](https://mlsanity.com/paper/2110.15238/) (2021)
- [LUT Tensor Core: A Software-Hardware Co-Design for LUT-Based Low-Bit LLM Inference](https://mlsanity.com/paper/2408.06003/) (2024)
- [The CoRa Tensor Compiler: Compilation for Ragged Tensors with Minimal Padding](https://mlsanity.com/paper/2110.10221/) (2021)
- [1xN Pattern for Pruning Convolutional Neural Networks](https://mlsanity.com/paper/2105.14713/) (2021)
- [Lite Pose: Efficient Architecture Design for 2D Human Pose Estimation](https://mlsanity.com/paper/2205.01271/) (2022)
- [TpuGraphs: A Performance Prediction Dataset on Large Tensor Computational Graphs](https://mlsanity.com/paper/2308.13490/) (2023)
- [Allo: A Programming Model for Composable Accelerator Design](https://mlsanity.com/paper/2404.04815/) (2024)
- [Adaptive Self-improvement LLM Agentic System for ML Library Development](https://mlsanity.com/paper/2502.02534/) (2025)

## Abstract (arXiv)

High-performance tensor programs are crucial to guarantee efficient execution of deep neural networks. However, obtaining performant tensor programs for different operators on various hardware platforms is notoriously challenging. Currently, deep learning systems rely on vendor-provided kernel libraries or various search strategies to get performant tensor programs. These approaches either require significant engineering effort to develop platform-specific optimization code or fall short of finding high-performance programs due to restricted search space and ineffective exploration strategy. We present Ansor, a tensor program generation framework for deep learning applications. Compared with existing search strategies, Ansor explores many more optimization combinations by sampling programs from a hierarchical representation of the search space. Ansor then fine-tunes the sampled programs with evolutionary search and a learned cost model to identify the best programs. Ansor can find high-performance programs that are outside the search space of existing state-of-the-art approaches. In addition, Ansor utilizes a task scheduler to simultaneously optimize multiple subgraphs in deep neural networks. We show that Ansor improves the execution performance of deep neural networks relative to the state-of-the-art on the Intel CPU, ARM CPU, and NVIDIA GPU by up to 3.8×, 2.6×, and 1.7×, respectively.