# LLM-Aided Compilation for Tensor Accelerators

arXiv: https://arxiv.org/abs/2408.03408 · HTML: https://mlsanity.com/paper/2408.03408/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 48 references that are in the map.
- First posted on arXiv 6 Aug 2024.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [Ansor: Generating High-Performance Tensor Programs for Deep Learning](https://mlsanity.com/paper/2006.06762/) (2020)
- [Gemmini: Enabling Systematic Deep-Learning Architecture Evaluation via Full-Stack Integration](https://mlsanity.com/paper/1911.09925/) (2019)
- [Large Language Models for Compiler Optimization](https://mlsanity.com/paper/2309.07062/) (2023)
- [TinyMPC: Model-Predictive Control on Resource-Constrained Microcontrollers](https://mlsanity.com/paper/2310.16985/) (2023)
- [Automated Transpilation of Imperative to Functional Code using Neural-Guided Program Synthesis (Extended Version)](https://mlsanity.com/paper/2203.09452/) (2022)
- [GPTPU: Accelerating Applications using Edge Tensor Processing Units](https://mlsanity.com/paper/2107.05473/) (2021)
- [Guess & Sketch: Language Model Guided Transpilation](https://mlsanity.com/paper/2309.14396/) (2023)
- [SIMD2: A Generalized Matrix Instruction Set for Accelerating Tensor Computation beyond GEMM](https://mlsanity.com/paper/2205.01252/) (2022)
- [MADNESS: A Multiresolution, Adaptive Numerical Environment for Scientific Simulation](https://mlsanity.com/paper/1507.01888/) (2015)

## Built on by (most cited first, 8 of 8)

- [Autocomp: A Powerful and Portable Code Optimizer for Tensor Accelerators](https://mlsanity.com/paper/2505.18574/) (2025)
- [hdl2v: A Code Translation Dataset for Enhanced LLM Verilog Generation](https://mlsanity.com/paper/2506.04544/) (2025)
- [Industrial LLM-based Code Optimization under Regulation: A Mixture-of-Agents Approach](https://mlsanity.com/paper/2508.03329/) (2025)
- [P4OMP: Retrieval-Augmented Prompting for OpenMP Parallelism in Serial Code](https://mlsanity.com/paper/2506.22703/) (2025)
- [Iceberg: Enhancing HLS Modeling with Synthetic Data](https://mlsanity.com/paper/2507.09948/) (2025)
- [ELK: Exploring the Efficiency of Inter-core Connected AI Chips with Deep Learning Compiler Techniques](https://mlsanity.com/paper/2507.11506/) (2025)
- [Exploring the Feasibility of End-to-End Large Language Model as a Compiler](https://mlsanity.com/paper/2511.04132/) (2025)
- [LACE: Large Language Model Aided Multi-Agent Framework for Agile RISC-V Instruction Extension](https://mlsanity.com/paper/2608.02915/) (2026)

## Abstract (arXiv)

Hardware accelerators, in particular accelerators for tensor processing, have many potential application domains. However, they currently lack the software infrastructure to support the majority of domains outside of deep learning. Furthermore, a compiler that can easily be updated to reflect changes at both application and hardware levels would enable more agile development and design space exploration of accelerators, allowing hardware designers to realize closer-to-optimal performance. In this work, we discuss how large language models (LLMs) could be leveraged to build such a compiler. Specifically, we demonstrate the ability of GPT-4 to achieve high pass rates in translating code to the Gemmini accelerator, and prototype a technique for decomposing translation into smaller, more LLM-friendly steps. Additionally, we propose a 2-phase workflow for utilizing LLMs to generate hardware-optimized code.