mlsanityData to 7 Oct 2026

Meta Large Language Model Compiler: Foundation Models of Compiler Optimization

arXiv 2407.02524 ↗First posted 27 Jun 2024cs.PL cs.AIcited by 39

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable capabilities across a variety of software engineering and coding tasks. However, their application in the domain of code and compiler optimization remains underexplored. Training LLMs is resource-intensive, requiring substantial GPU hours and extensive data collection, which can be prohibitive. To address this gap, we introduce Meta Large Language Model Compiler (LLM Compiler), a suite of robust, openly available, pre-trained models specifically designed for code optimization tasks. Built on the foundation of Code Llama, LLM Compiler enhances the understanding of compiler intermediate representations (IRs), assembly language, and optimization techniques. The model has been trained on a vast corpus of 546 billion tokens of LLVM-IR and assembly code and has undergone instruction fine-tuning to interpret compiler behavior. LLM Compiler is released under a bespoke commercial license to allow wide reuse and is available in two sizes: 7 billion and 13 billion parameters. We also present fine-tuned versions of the model, demonstrating its enhanced capabilities in optimizing code size and disassembling from x86_64 and ARM assembly back into LLVM-IR. These achieve 77% of the optimising potential of an autotuning search, and 45% disassembly round trip (14% exact match). This release aims to provide a scalable, cost-effective foundation for further research and development in compiler optimization by both academic researchers and industry practitioners.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 40 of its 60 references that are in the map. First posted on arXiv 27 Jun 2024.

Builds on

40 of 60 references.

References · 40

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  7. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  8. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  9. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  10. 2018Model Cards for Model Reportingcited by 1,068
  11. 2024A Survey on Large Language Models for Code Generationcited by 646
  12. 2023Textbooks Are All You Needcited by 561
28 more
  1. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  2. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 484
  3. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  4. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 327
  5. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 294
  6. 2020Unsupervised Translation of Programming Languagescited by 262
  7. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 251
  8. 2023LongNet: Scaling Transformers to 1,000,000,000 Tokenscited by 201
  9. 2023SantaCoder: don't reach for the stars!cited by 169
  10. 2022Practical Program Repair in the Era of Large Pre-trained Language Modelscited by 133
  11. 2023Learning Performance-Improving Code Editscited by 103
  12. 2022Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Modelscited by 102
  13. 2022Code Translation with Compiler Representationscited by 42
  14. 2021CompilerGym: Robust, Performant Compiler Optimization Environments for AI Researchcited by 41
  15. 2021MLGO: a Machine Learning Guided Compiler Optimizations Frameworkcited by 40
  16. 2023SLaDe: A Portable Small Language Model Decompiler for Optimized Assemblycited by 28
  17. 2024LLM-Vectorizer: LLM-based Verified Loop Vectorizercited by 25
  18. 2024IRCoder: Intermediate Representations Make Language Models Robust Multilingual Code Generatorscited by 21
  19. 2022Beyond the C: Retargetable Decompilation using Neural Machine Translationcited by 15
  20. 2018Machine Learning in Compiler Optimisationcited by 13
  21. 2022Cross-Language Binary-Source Code Matching with Intermediate Representationscited by 13
  22. 2021Learning C to x86 Translation: An Experiment in Neural Compilationcited by 12
  23. 2023Boosting Neural Networks to Decompile Optimized Binariescited by 11
  24. 2023Learning Compiler Pass Orders using Coreset and Normalized Value Predictioncited by 11
  25. 2023ComPile: A Large IR Dataset from Production Sourcescited by 10
  26. 2022MLGOPerf: An ML Guided Inliner to Optimize Performancecited by 9
  27. 2024Compiler generated feedback for Large Language Modelscited by 9
  28. 2024Priority Sampling of Large Language Models for Compilerscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 24; 2026: 11.

39 papers.

Sort:

By area:

  1. 2025CWM: An Open-Weights LLM for Research on Code Generation with World Modelscited by 87
  2. 2025AlgoTune: Can Language Models Speed Up General-Purpose Numerical Programs?cited by 24
  3. 2025EquiBench: Benchmarking Large Language Models' Reasoning about Program Semantics via Equivalence Checkingcited by 23
  4. 2025cAST: Enhancing Code Retrieval-Augmented Generation with Structural Chunking via Abstract Syntax Treecited by 17
  5. 2025Reasoning Beyond Limits: Advances and Open Problems for LLMscited by 16
  6. 2025Autocomp: A Powerful and Portable Code Optimizer for Tensor Acceleratorscited by 12
  7. 2024Don't Transform the Code, Code the Transforms: Towards Precise Code Rewriting using LLMscited by 8
  8. 2026ShieldedCode: Learning Robust Representations for Virtual Machine Protected Codecited by 8
  9. 2024MetRex: A Benchmark for Verilog Code Metric Reasoning Using LLMscited by 7
  10. 2025Compiler-R1: Towards Agentic Compiler Auto-tuning with Reinforcement Learningcited by 7
  11. 2025Tuning LLM-based Code Optimization via Meta-Prompting: An Industrial Perspectivecited by 7
  12. 2025Differentiation-Based Extraction of Proprietary Data from Fine-Tuned LLMscited by 6
27 more
  1. 2025SuperCoder: Assembly Program Superoptimization with Large Language Modelscited by 5
  2. 2026Magellan: Autonomous Discovery of Novel Compiler Optimization Heuristics with AlphaEvolvecited by 5
  3. 2025ASMA-Tune: Unlocking LLMs' Assembly Code Comprehension via Structural-Semantic Instruction Tuningcited by 4
  4. 2025AwareCompiler: Agentic Context-Aware Compiler Optimization via a Synergistic Knowledge-Data Driven Frameworkcited by 4
  5. 2025Agentic Auto-Scheduling: An Experimental Study of LLM-Guided Loop Optimizationcited by 4
  6. 2025Fine-tuned LLM-based Code Migration Frameworkcited by 2
  7. 2026Credit-Budgeted ICPC-Style Coding: When Agents Must Pay for Every Decisioncited by 2
  8. 2024Tadashi: Enabling AI-Based Automated Code Generation With Guaranteed Correctnesscited by 1
  9. 2024Towards LLM-based optimization compilers. Can LLMs learn how to apply a single peephole optimization? Reasoning is all LLMs need!cited by 1
  10. 2025Can Large Language Models Understand Intermediate Representations in Compilers?cited by 1
  11. 2025Slim attention: cut your context memory in half without loss -- K-cache is all you need for MHAcited by 1
  12. 2025Omniwise: Predicting GPU Kernels Performance with LLMscited by 1
  13. 2025Rethinking LLM-Based RTL Code Optimization Via Timing Logic Metamorphosiscited by 1
  14. 2025LOOPRAG: Enhancing Loop Transformation Optimization with Retrieval-Augmented Large Language Modelscited by 1
  15. 2026AutoPass: Evidence-Guided LLM Agents for Compiler Performance Tuningcited by 1
  16. 2025Guaranteed Guess: A Language Modeling Approach for CISC-to-RISC Transpilation with Testing Guaranteescited by 0
  17. 2025CRINN: Contrastive Reinforcement Learning for Approximate Nearest Neighbor Searchcited by 0
  18. 2025QiMeng-NeuComBack: Self-Evolving Translation from IR to Assembly Codecited by 0
  19. 2025Exploring the Feasibility of End-to-End Large Language Model as a Compilercited by 0
  20. 2025C-ing Clearly: Enhanced Binary Code Explanations using C codecited by 0
  21. 2026ParaCodex: A Profiling-Guided Autonomous Coding Agent for Reliable Parallel Code Generation and Translationcited by 0
  22. 2026ECCO: Evidence-Driven Causal Reasoning for Compiler Optimizationcited by 0
  23. 2026LLM-Guided Strategy Synthesis for Scalable Equality Saturationcited by 0
  24. 2026Step-TP: A Grounded, Step-Level Dataset with Chain-of-Thought Reasoning for LLM-Guided Tensor Program Optimizationcited by 0
  25. 2026The Biomimetic Architecture of Software 4.0cited by 0
  26. 2026LLM4LLM: Bridging Kernel Benchmarks and Real Deployment via Closed-Loop Agentic Optimizationcited by 0
  27. 2026CompileRover: Revolutionizing Virtual Machine Compiler Optimization with a Tri-Role LLM-Driven Frameworkcited by 0