# Compiling ONNX Neural Network Models Using MLIR

arXiv: https://arxiv.org/abs/2008.08272 · HTML: https://mlsanity.com/paper/2008.08272/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Aug 2020.

## Built on by (most cited first, 18 of 18)

- [A Large-Scale Exploit Instrumentation Study of AI/ML Supply Chain Attacks in Hugging Face Models](https://mlsanity.com/paper/2410.04490/) (2024)
- [Coverage-Guided Tensor Compiler Fuzzing with Joint IR-Pass Mutation](https://mlsanity.com/paper/2202.09947/) (2022)
- [Neural Brain: A Neuroscience-inspired Framework for Embodied Agents](https://mlsanity.com/paper/2505.07634/) (2025)
- [An Empirical Study of Challenges in Converting Deep Learning Models](https://mlsanity.com/paper/2206.14322/) (2022)
- [Mind Meets Space: Rethinking Agentic Spatial Intelligence from a Neuroscience-inspired Perspective](https://mlsanity.com/paper/2509.09154/) (2025)
- [TPU-MLIR: A Compiler For TPU Using MLIR](https://mlsanity.com/paper/2210.15016/) (2022)
- [Advancing Direct Convolution using Convolution Slicing Optimization and ISA Extensions](https://mlsanity.com/paper/2303.04739/) (2023)
- [A Reinforcement Learning Environment for Automatic Code Optimization in the MLIR Compiler](https://mlsanity.com/paper/2409.11068/) (2024)
- [Hydraulis: Balancing Large Transformer Model Training via Co-designing Parallel Strategies and Data Assignment](https://mlsanity.com/paper/2412.07894/) (2024)
- [CyberCortex.AI: An AI-based Operating System for Autonomous Robotics and Complex Automation](https://mlsanity.com/paper/2409.01241/) (2024)
- [Chat-Ghosting: A Comparative Study of Methods for Auto-Completion in Dialog Systems](https://mlsanity.com/paper/2507.05940/) (2025)
- [Bootstrapping Fuzzers for Compilers of Low-Resource Language Dialects Using Language Models](https://mlsanity.com/paper/2512.05887/) (2025)
- [Tensor Slicing and Optimization for Multicore NPUs](https://mlsanity.com/paper/2304.03013/) (2023)
- [A Survey on Design Methodologies for Accelerating Deep Learning on Heterogeneous Architectures](https://mlsanity.com/paper/2311.17815/) (2023)
- [Explore as a Storm, Exploit as a Raindrop: On the Benefit of Fine-Tuning Kernel Schedulers with Coordinate Descent](https://mlsanity.com/paper/2406.20037/) (2024)
- [DejAIvu: Identifying and Explaining AI Art on the Web in Real-Time with Saliency Maps](https://mlsanity.com/paper/2502.08821/) (2025)
- [Rulebook: bringing co-routines to reinforcement learning environments](https://mlsanity.com/paper/2504.19625/) (2025)
- [ShadowLogic: Backdoors in Any Whitebox LLM](https://mlsanity.com/paper/2511.00664/) (2025)

## Abstract (arXiv)

Deep neural network models are becoming increasingly popular and have been used in various tasks such as computer vision, speech recognition, and natural language processing. Machine learning models are commonly trained in a resource-rich environment and then deployed in a distinct environment such as high availability machines or edge devices. To assist the portability of models, the open-source community has proposed the Open Neural Network Exchange (ONNX) standard. In this paper, we present a high-level, preliminary report on our onnx-mlir compiler, which generates code for the inference of deep neural network models described in the ONNX format. Onnx-mlir is an open-source compiler implemented using the Multi-Level Intermediate Representation (MLIR) infrastructure recently integrated in the LLVM project. Onnx-mlir relies on the MLIR concept of dialects to implement its functionality. We propose here two new dialects: (1) an ONNX specific dialect that encodes the ONNX standard semantics, and (2) a loop-based dialect to provide for a common lowering point for all ONNX dialect operations. Each intermediate representation facilitates its own characteristic set of graph-level and loop-based optimizations respectively. We illustrate our approach by following several models through the proposed representations and we include some early optimization work and performance results.