# Compiler generated feedback for Large Language Models

arXiv: https://arxiv.org/abs/2403.14714 · HTML: https://mlsanity.com/paper/2403.14714/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 39 references that are in the map.
- First posted on arXiv 18 Mar 2024.

## Builds on (most cited first, 21 of 21)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022)
- [Unsupervised Translation of Programming Languages](https://mlsanity.com/paper/2006.03511/) (2020)
- [Practical Program Repair in the Era of Large Pre-trained Language Models](https://mlsanity.com/paper/2210.14179/) (2022)
- [Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models](https://mlsanity.com/paper/2212.14834/) (2022)
- [Large Language Models for Compiler Optimization](https://mlsanity.com/paper/2309.07062/) (2023)
- [Code Translation with Compiler Representations](https://mlsanity.com/paper/2207.03578/) (2022)
- [MLGO: a Machine Learning Guided Compiler Optimizations Framework](https://mlsanity.com/paper/2101.04808/) (2021)
- [NeuroVectorizer: End-to-End Vectorization with Deep Reinforcement Learning](https://mlsanity.com/paper/1909.13639/) (2019)
- [AutoPhase: Juggling HLS Phase Orderings in Random Forests with Deep Reinforcement Learning](https://mlsanity.com/paper/2003.00671/) (2020)
- [Fixing Hardware Security Bugs with Large Language Models](https://mlsanity.com/paper/2302.01215/) (2023)
- [Machine Learning in Compiler Optimisation](https://mlsanity.com/paper/1805.03441/) (2018)
- [Automated Conformance Testing for JavaScript Engines via Deep Compiler Fuzzing](https://mlsanity.com/paper/2104.07460/) (2021)
- [Learning Compiler Pass Orders using Coreset and Normalized Value Prediction](https://mlsanity.com/paper/2301.05104/) (2023)
- [MLGOPerf: An ML Guided Inliner to Optimize Performance](https://mlsanity.com/paper/2207.08389/) (2022)

## Built on by (most cited first, 9 of 9)

- [Meta Large Language Model Compiler: Foundation Models of Compiler Optimization](https://mlsanity.com/paper/2407.02524/) (2024)
- [Towards a Unified View of Preference Learning for Large Language Models: A Survey](https://mlsanity.com/paper/2409.02795/) (2024)
- [SuperCoder: Assembly Program Superoptimization with Large Language Models](https://mlsanity.com/paper/2505.11480/) (2025)
- [Prompt to Pwn: Automated Exploit Generation for Smart Contracts](https://mlsanity.com/paper/2508.01371/) (2025)
- [Agentic Auto-Scheduling: An Experimental Study of LLM-Guided Loop Optimization](https://mlsanity.com/paper/2511.00592/) (2025)
- [Towards Secure Program Partitioning for Smart Contracts with LLM's In-Context Learning](https://mlsanity.com/paper/2502.14215/) (2025)
- [LOOPRAG: Enhancing Loop Transformation Optimization with Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2512.15766/) (2025)
- [From LLMs to Agents in Programming: The Impact of Providing an LLM with a Compiler](https://mlsanity.com/paper/2601.12146/) (2026)
- [Natural-Language to SysMLv2 Translation via Conformance-Driven Iterative Refinement](https://mlsanity.com/paper/2607.14162/) (2026)

## Abstract (arXiv)

We introduce a novel paradigm in compiler optimization powered by Large Language Models with compiler feedback to optimize the code size of LLVM assembly. The model takes unoptimized LLVM IR as input and produces optimized IR, the best optimization passes, and instruction counts of both unoptimized and optimized IRs. Then we compile the input with generated optimization passes and evaluate if the predicted instruction count is correct, generated IR is compilable, and corresponds to compiled code. We provide this feedback back to LLM and give it another chance to optimize code. This approach adds an extra 0.53% improvement over -Oz to the original model. Even though, adding more information with feedback seems intuitive, simple sampling techniques achieve much higher performance given 10 or more samples.