# LLM-Vectorizer: LLM-based Verified Loop Vectorizer

arXiv: https://arxiv.org/abs/2406.04693 · HTML: https://mlsanity.com/paper/2406.04693/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 27 references that are in the map.
- First posted on arXiv 7 Jun 2024.

## Builds on (most cited first, 6 of 6)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Unsupervised Translation of Programming Languages](https://mlsanity.com/paper/2006.03511/) (2020)
- [Large Language Models for Compiler Optimization](https://mlsanity.com/paper/2309.07062/) (2023)
- [NeuroVectorizer: End-to-End Vectorization with Deep Reinforcement Learning](https://mlsanity.com/paper/1909.13639/) (2019)
- [3DGen: AI-Assisted Generation of Provably Correct Binary Format Parsers](https://mlsanity.com/paper/2404.10362/) (2024)

## Built on by (most cited first, 25 of 25)

- [Astra: A Multi-Agent System for GPU Kernel Performance Optimization](https://mlsanity.com/paper/2509.07506/) (2025)
- [Meta Large Language Model Compiler: Foundation Models of Compiler Optimization](https://mlsanity.com/paper/2407.02524/) (2024)
- [The FM Agent](https://mlsanity.com/paper/2510.26144/) (2025)
- [Autocomp: A Powerful and Portable Code Optimizer for Tensor Accelerators](https://mlsanity.com/paper/2505.18574/) (2025)
- [KernelBlaster: Continual Cross-Task CUDA Optimization via Memory-Augmented In-Context Reinforcement Learning](https://mlsanity.com/paper/2602.14293/) (2026)
- [ProofWright: Towards Agentic Formal Verification of CUDA](https://mlsanity.com/paper/2511.12294/) (2025)
- [UNO Arena for Evaluating Sequential Decision-Making Capability of Large Language Models](https://mlsanity.com/paper/2406.16382/) (2024)
- [SuperCoder: Assembly Program Superoptimization with Large Language Models](https://mlsanity.com/paper/2505.11480/) (2025)
- [VecTrans: Enhancing Compiler Auto-Vectorization through LLM-Assisted Code Transformations](https://mlsanity.com/paper/2503.19449/) (2025)
- [Agentic Auto-Scheduling: An Experimental Study of LLM-Guided Loop Optimization](https://mlsanity.com/paper/2511.00592/) (2025)
- [PerfCoder: Large Language Models for Interpretable Code Performance Optimization](https://mlsanity.com/paper/2512.14018/) (2025)
- [GPU Kernel Optimization Beyond Full Builds: An LLM Framework with Minimal Executable Programs](https://mlsanity.com/paper/2512.22147/) (2025)
- [A Two-Stage GPU Kernel Tuner Combining Semantic Refactoring and Search-Based Optimization](https://mlsanity.com/paper/2601.12698/) (2026)
- [Agentic Harness for Real-World Compilers](https://mlsanity.com/paper/2603.20075/) (2026)
- [Beyond Pass-by-Pass Optimization: Intent-Driven IR Optimization with Large Language Models](https://mlsanity.com/paper/2602.18511/) (2026)
- [LOOPRAG: Enhancing Loop Transformation Optimization with Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2512.15766/) (2025)
- [Rudder: Steering Prefetching in Distributed GNN Training using LLM Agents](https://mlsanity.com/paper/2602.23556/) (2026)
- [AutoPass: Evidence-Guided LLM Agents for Compiler Performance Tuning](https://mlsanity.com/paper/2606.20373/) (2026)
- [Instruction Set Migration at Warehouse Scale](https://mlsanity.com/paper/2510.14928/) (2025)
- [The New Compiler Stack: A Survey on the Synergy of LLMs and Compilers](https://mlsanity.com/paper/2601.02045/) (2026)
- [AutoVecCoder: Teaching LLMs to Generate Explicitly Vectorized Code](https://mlsanity.com/paper/2605.17978/) (2026)
- [Semantic-aware and Self-improving Program Reduction via Agentic Large Language Models](https://mlsanity.com/paper/2607.03766/) (2026)
- [T-LLM Compiler: Trusted LLM-based Code Optimization and Verification Framework](https://mlsanity.com/paper/2608.14953/) (2026)
- [Simthesizer: An Agent-Driven Simulation Framework for LLM Serving Systems](https://mlsanity.com/paper/2608.24650/) (2026)
- [Do AI Agents Understand Computer Architecture?](https://mlsanity.com/paper/2609.19387/) (2026)

## Abstract (arXiv)

Vectorization is a powerful optimization technique that significantly boosts the performance of high performance computing applications operating on large data arrays. Despite decades of research on auto-vectorization, compilers frequently miss opportunities to vectorize code. On the other hand, writing vectorized code manually using compiler intrinsics is still a complex, error-prone task that demands deep knowledge of specific architecture and compilers. In this paper, we evaluate the potential of large-language models (LLMs) to generate vectorized (Single Instruction Multiple Data) code from scalar programs that process individual array elements. We propose a novel finite-state machine multi-agents based approach that harnesses LLMs and test-based feedback to generate vectorized code. Our findings indicate that LLMs are capable of producing high performance vectorized code with run-time speedup ranging from 1.1x to 9.4x as compared to the state-of-the-art compilers such as Intel Compiler, GCC, and Clang. To verify the correctness of vectorized code, we use Alive2, a leading bounded translation validation tool for LLVM IR. We describe a few domain-specific techniques to improve the scalability of Alive2 on our benchmark dataset. Overall, our approach is able to verify 38.2% of vectorizations as correct on the TSVC benchmark dataset.