# Relay: A High-Level Compiler for Deep Learning

arXiv: https://arxiv.org/abs/1904.08368 · HTML: https://mlsanity.com/paper/1904.08368/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2019.

## Built on by (most cited first, 4 of 4)

- [vMCU: Coordinated Memory Management and Kernel Optimization for DNN Inference on MCUs](https://mlsanity.com/paper/2406.06542/) (2024)
- [Quantization in Layer's Input is Matter](https://mlsanity.com/paper/2202.05137/) (2022)
- [Mixed-Precision Inference Quantization: Radically Towards Faster inference speed, Lower Storage requirement, and Lower Loss](https://mlsanity.com/paper/2207.10083/) (2022)
- [FTuner: A Fast Dynamic Shape Tensors Program Auto-Tuner for Deep Learning Compilers](https://mlsanity.com/paper/2407.21418/) (2024)

## Abstract (arXiv)

Frameworks for writing, compiling, and optimizing deep learning (DL) models have recently enabled progress in areas like computer vision and natural language processing. Extending these frameworks to accommodate the rapidly diversifying landscape of DL models and hardware platforms presents challenging tradeoffs between expressivity, composability, and portability. We present Relay, a new compiler framework for DL. Relay's functional, statically typed intermediate representation (IR) unifies and generalizes existing DL IRs to express state-of-the-art models. The introduction of Relay's expressive IR requires careful design of domain-specific optimizations, addressed via Relay's extension mechanisms. Using these extension mechanisms, Relay supports a unified compiler that can target a variety of hardware platforms. Our evaluation demonstrates Relay's competitive performance for a broad class of models and devices (CPUs, GPUs, and emerging accelerators). Relay's design demonstrates how a unified IR can provide expressivity, composability, and portability without compromising performance.