# OMPar: Automatic Parallelization with AI-Driven Source-to-Source Compilation

arXiv: https://arxiv.org/abs/2409.14771 · HTML: https://mlsanity.com/paper/2409.14771/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 53 references that are in the map.
- First posted on arXiv 23 Sep 2024.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Can Large Language Models Write Parallel Code?](https://mlsanity.com/paper/2401.12554/) (2024)
- [Comparing Llama-2 and GPT-3 LLMs for HPC kernels generation](https://mlsanity.com/paper/2309.07103/) (2023)
- [The Landscape and Challenges of HPC Research and LLMs](https://mlsanity.com/paper/2402.02018/) (2024)
- [OMPGPT: A Generative Pre-trained Transformer Model for OpenMP](https://mlsanity.com/paper/2401.16445/) (2024)
- [MonoCoder: Domain-Specific Code Language Model for HPC Codes and Tasks](https://mlsanity.com/paper/2312.13322/) (2023)
- [HPC-Coder: Modeling Parallel Programs using Large Language Models](https://mlsanity.com/paper/2306.17281/) (2023)
- [Learning to Parallelize in a Shared-Memory Environment with Transformers](https://mlsanity.com/paper/2204.12835/) (2022)
- [Advising OpenMP Parallelization via a Graph-Based Approach with Transformers](https://mlsanity.com/paper/2305.11999/) (2023)
- [ComPar: Optimized Multi-Compiler for Automatic OpenMP S2S Parallelization](https://mlsanity.com/paper/2005.13304/) (2020)
- [Specifying and Executing Optimizations for Parallel Programs](https://mlsanity.com/paper/1407.7932/) (2014)

## Built on by (most cited first, 5 of 5)

- [P4OMP: Retrieval-Augmented Prompting for OpenMP Parallelism in Serial Code](https://mlsanity.com/paper/2506.22703/) (2025)
- [Accelerating Latency-Critical Applications with AI-Powered Semi-Automatic Fine-Grained Parallelization on SMT Processors](https://mlsanity.com/paper/2509.00883/) (2025)
- [The New Compiler Stack: A Survey on the Synergy of LLMs and Compilers](https://mlsanity.com/paper/2601.02045/) (2026)
- [ParaCodex: A Profiling-Guided Autonomous Coding Agent for Reliable Parallel Code Generation and Translation](https://mlsanity.com/paper/2601.04327/) (2026)
- [HARGO: Heterogeneity-Aware Reward-Guided Optimization for RL Post-Training of LLMs on HPC Tasks](https://mlsanity.com/paper/2607.28301/) (2026)

## Abstract (arXiv)

Manual parallelization of code remains a significant challenge due to the complexities of modern software systems and the widespread adoption of multi-core architectures. This paper introduces OMPar, an AI-driven tool designed to automate the parallelization of C/C++ code using OpenMP pragmas. OMPar integrates Large Language Models (LLMs) through two key components: OMPify, which assesses loop parallelization potential, and MonoCoder-OMP, a new fine-tuned model which generates precise OpenMP pragmas. The evaluation of OMPar follows the same rigorous process applied to traditional tools like source-to-source AutoPar and ICPC compilers: (1) ensuring the generated code compiles and runs correctly in serial form, (2) assessing performance with the gradual addition of threads and corresponding physical cores, and (3) verifying and validating the correctness of the code's output. Benchmarks from HeCBench and ParEval are used to evaluate accuracy and performance. Experimental results demonstrate that OMPar significantly outperforms traditional methods, achieving higher accuracy in identifying parallelizable loops and generating efficient pragmas. Beyond accuracy, OMPar offers advantages such as the ability to work on partial or incomplete codebases and the capacity to continuously learn from new code patterns, enhancing its parallelization capabilities over time. These results underscore the potential of LLMs in revolutionizing automatic parallelization techniques, paving the way for more efficient and scalable parallel computing systems.