# ComPile: A Large IR Dataset from Production Sources

arXiv: https://arxiv.org/abs/2309.15432 · HTML: https://mlsanity.com/paper/2309.15432/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2023.

## Built on by (most cited first, 10 of 10)

- [Meta Large Language Model Compiler: Foundation Models of Compiler Optimization](https://mlsanity.com/paper/2407.02524/) (2024)
- [IRCoder: Intermediate Representations Make Language Models Robust Multilingual Code Generators](https://mlsanity.com/paper/2403.03894/) (2024)
- [MonoCoder: Domain-Specific Code Language Model for HPC Codes and Tasks](https://mlsanity.com/paper/2312.13322/) (2023)
- [Offline Imitation Learning from Multiple Baselines with Applications to Compiler Optimization](https://mlsanity.com/paper/2403.19462/) (2024)
- [Focus on Likely Classes for Test-Time Prediction](https://mlsanity.com/paper/2505.03819/) (2025)
- [PassNet: Scaling Large Language Models for Graph Compiler Pass Generation](https://mlsanity.com/paper/2605.29357/) (2026)
- [Modeling Code: Is Text All You Need?](https://mlsanity.com/paper/2507.11467/) (2025)
- [Exploring the Feasibility of End-to-End Large Language Model as a Compiler](https://mlsanity.com/paper/2511.04132/) (2025)
- [LLM Translation of Compiler Intermediate Representation](https://mlsanity.com/paper/2605.08247/) (2026)
- [Beyond the Need for Speed: Energy-Aware Code Generation via Simulation-Guided Reinforcement Learning](https://mlsanity.com/paper/2607.04577/) (2026)

## Abstract (arXiv)

Code is increasingly becoming a core data modality of modern machine learning research impacting not only the way we write code with conversational agents like OpenAI's ChatGPT, Google's Bard, or Anthropic's Claude, the way we translate code from one language into another, but also the compiler infrastructure underlying the language. While modeling approaches may vary and representations differ, the targeted tasks often remain the same within the individual classes of models. Relying solely on the ability of modern models to extract information from unstructured code does not take advantage of 70 years of programming language and compiler development by not utilizing the structure inherent to programs in the data collection. This detracts from the performance of models working over a tokenized representation of input code and precludes the use of these models in the compiler itself. To work towards the first intermediate representation (IR) based models, we fully utilize the LLVM compiler infrastructure, shared by a number of languages, to generate a 182B token dataset of LLVM IR. We generated this dataset from programming languages built on the shared LLVM infrastructure, including Rust, Swift, Julia, and C/C++, by hooking into LLVM code generation either through the language's package manager or the compiler directly to extract the dataset of intermediate representations from production grade programs. Statistical analysis proves the utility of our dataset not only for large language model training, but also for the introspection into the code generation process itself with the dataset showing great promise for machine-learned compiler components.