# oneDNN Graph Compiler: A Hybrid Approach for High-Performance Deep Learning Compilation

arXiv: https://arxiv.org/abs/2301.01333 · HTML: https://mlsanity.com/paper/2301.01333/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 3 Jan 2023.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [Ansor: Generating High-Performance Tensor Programs for Deep Learning](https://mlsanity.com/paper/2006.06762/) (2020)
- [Learning to Optimize Tensor Programs](https://mlsanity.com/paper/1805.08166/) (2018)
- [Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractions](https://mlsanity.com/paper/1802.04730/) (2018)
- [MLIR: A Compiler Infrastructure for the End of Moore's Law](https://mlsanity.com/paper/2002.11054/) (2020)
- [The Deep Learning Compiler: A Comprehensive Survey](https://mlsanity.com/paper/2002.03794/) (2020)
- [Tensor Processing Primitives: A Programming Abstraction for Efficiency and Portability in Deep Learning & HPC Workloads](https://mlsanity.com/paper/2104.05755/) (2021)
- [Composable and Modular Code Generation in MLIR: A Structured and Retargetable Approach to Tensor Compiler Construction](https://mlsanity.com/paper/2202.03293/) (2022)
- [MIOpen: An Open Source Library For Deep Learning Primitives](https://mlsanity.com/paper/1910.00078/) (2019)
- [Stripe: Tensor Compilation via the Nested Polyhedral Model](https://mlsanity.com/paper/1903.06498/) (2019)
- [PolyDL: Polyhedral Optimizations for Creation of High Performance DL primitives](https://mlsanity.com/paper/2006.02230/) (2020)

## Built on by (most cited first, 7 of 7)

- [TorchTitan: One-stop PyTorch native solution for production ready LLM pre-training](https://mlsanity.com/paper/2410.06511/) (2024)
- [Harnessing Deep Learning and HPC Kernels via High-Level Loop and Tensor Abstractions on CPU Architectures](https://mlsanity.com/paper/2304.12576/) (2023)
- [Towards a high-performance AI compiler with upstream MLIR](https://mlsanity.com/paper/2404.15204/) (2024)
- [MAS-Attention: Memory-Aware Stream Processing for Attention Acceleration on Resource-Constrained Edge Devices](https://mlsanity.com/paper/2411.17720/) (2024)
- [Space Filling Curves is All You Need: Communication-Avoiding Matrix Multiplication Made Simple](https://mlsanity.com/paper/2601.16294/) (2026)
- [LP-GEMM: Integrating Layout Propagation into GEMM Operations](https://mlsanity.com/paper/2604.04599/) (2026)
- [Tensor Seeks Layout: Formalizing Layout Selection for ML Compilers](https://mlsanity.com/paper/2608.21555/) (2026)

## Abstract (arXiv)

With the rapid development of deep learning models and hardware support for dense computing, the deep learning workload characteristics changed significantly from a few hot spots on compute-intensive operations to a broad range of operations scattered across the models. Accelerating a few compute-intensive operations using the expert-tuned implementation of primitives does not fully exploit the performance potential of AI hardware. Various efforts have been made to compile a full deep neural network (DNN) graph. One of the biggest challenges is to achieve high-performance tensor compilation by generating expert level performance code for the dense compute-intensive operations and applying compilation optimization at the scope of DNN computation graph across multiple compute-intensive operations. We present oneDNN Graph Compiler, a tensor compiler that employs a hybrid approach of using techniques from both compiler optimization and expert-tuned kernels for high performance code generation of the deep neural network graph. oneDNN Graph Compiler addresses unique optimization challenges in the deep learning domain, such as low-precision computation, aggressive fusion of graph operations, optimization for static tensor shapes and memory layout, constant weight optimization, and memory buffer reuse. Experimental results demonstrate significant performance gains over existing tensor compiler and primitives library for performance-critical DNN computation graphs and end-to-end models on Intel Xeon Scalable Processors.