# nncase: An End-to-End Compiler for Efficient LLM Deployment on Heterogeneous Storage Architectures

arXiv: https://arxiv.org/abs/2512.21571 · HTML: https://mlsanity.com/paper/2512.21571/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 58 references that are in the map.
- First posted on arXiv 25 Dec 2025.

## Builds on (most cited first, 11 of 11)

- [Ansor: Generating High-Performance Tensor Programs for Deep Learning](https://mlsanity.com/paper/2006.06762/) (2020)
- [Learning to Optimize Tensor Programs](https://mlsanity.com/paper/1805.08166/) (2018)
- [GSPMD: General and Scalable Parallelization for ML Computation Graphs](https://mlsanity.com/paper/2105.04663/) (2021)
- [Tensor Program Optimization with Probabilistic Programs](https://mlsanity.com/paper/2205.13603/) (2022)
- [Equality Saturation for Tensor Graph Superoptimization](https://mlsanity.com/paper/2101.01332/) (2021)
- [Pure Tensor Program Rewriting via Access Patterns (Representation Pearl)](https://mlsanity.com/paper/2105.09377/) (2021)
- [End-to-end Adaptive Distributed Training on PaddlePaddle](https://mlsanity.com/paper/2112.02752/) (2021)
- [Colossal-Auto: Unified Automation of Parallelization and Activation Checkpoint for Large-scale Models](https://mlsanity.com/paper/2302.02599/) (2023)
- [Tuna: A Static Analysis Approach to Optimizing Deep Neural Networks](https://mlsanity.com/paper/2104.14641/) (2021)
- [AutoDDL: Automatic Distributed Deep Learning with Near-Optimal Bandwidth Cost](https://mlsanity.com/paper/2301.06813/) (2023)
- [Optimizing Tensor Computation Graphs with Equality Saturation and Monte Carlo Tree Search](https://mlsanity.com/paper/2410.05534/) (2024)

## Abstract (arXiv)

The efficient deployment of large language models (LLMs) is hindered by memory architecture heterogeneity, where traditional compilers suffer from fragmented workflows and high adaptation costs. We present nncase, an open-source, end-to-end compilation framework designed to unify optimization across diverse targets. Central to nncase is an e-graph-based term rewriting engine that mitigates the phase ordering problem, enabling global exploration of computation and data movement strategies. The framework integrates three key modules: Auto Vectorize for adapting to heterogeneous computing units, Auto Distribution for searching parallel strategies with cost-aware communication optimization, and Auto Schedule for maximizing on-chip cache locality. Furthermore, a buffer-aware Codegen phase ensures efficient kernel instantiation. Evaluations show that nncase outperforms mainstream frameworks like MLC LLM and Intel IPEX on Qwen3 series models and achieves performance comparable to the hand-optimized llama.cpp on CPUs, demonstrating the viability of automated compilation for high-performance LLM deployment. The source code is available at https://github.com/kendryte/nncase.