# ParaCodex: A Profiling-Guided Autonomous Coding Agent for Reliable Parallel Code Generation and Translation

arXiv: https://arxiv.org/abs/2601.04327 · HTML: https://mlsanity.com/paper/2601.04327/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 50 references that are in the map.
- First posted on arXiv 7 Jan 2026.

## Builds on (most cited first, 30 of 33)

- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X](https://mlsanity.com/paper/2303.17568/) (2023)
- [Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation](https://mlsanity.com/paper/2310.02304/) (2023)
- [Learning to Optimize Tensor Programs](https://mlsanity.com/paper/1805.08166/) (2018)
- [Meta Large Language Model Compiler: Foundation Models of Compiler Optimization](https://mlsanity.com/paper/2407.02524/) (2024)
- [Can Large Language Models Write Parallel Code?](https://mlsanity.com/paper/2401.12554/) (2024)
- [Comparing Llama-2 and GPT-3 LLMs for HPC kernels generation](https://mlsanity.com/paper/2309.07103/) (2023)
- [Hidet: Task-Mapping Programming Paradigm for Deep Learning Tensor Programs](https://mlsanity.com/paper/2210.09603/) (2022)
- [PerfCodeGen: Improving Performance of LLM Generated Code with Execution Feedback](https://mlsanity.com/paper/2412.03578/) (2024)
- [Evaluation of OpenAI Codex for HPC Parallel Programming Models Kernel Generation](https://mlsanity.com/paper/2306.15121/) (2023)
- [The Landscape and Challenges of HPC Research and LLMs](https://mlsanity.com/paper/2402.02018/) (2024)
- [Agentic Refactoring: An Empirical Study of AI Coding Agents](https://mlsanity.com/paper/2511.04824/) (2025)
- [Learning to Parallelize with OpenMP by Augmented Heterogeneous AST Representation](https://mlsanity.com/paper/2305.05779/) (2023)
- [The Three Pillars of Machine Programming](https://mlsanity.com/paper/1803.07244/) (2018)
- [MonoCoder: Domain-Specific Code Language Model for HPC Codes and Tasks](https://mlsanity.com/paper/2312.13322/) (2023)
- [Learning to Parallelize in a Shared-Memory Environment with Transformers](https://mlsanity.com/paper/2204.12835/) (2022)
- [Advising OpenMP Parallelization via a Graph-Based Approach with Transformers](https://mlsanity.com/paper/2305.11999/) (2023)
- [LASSI: An LLM-based Automated Self-Correcting Pipeline for Translating Parallel Scientific Codes](https://mlsanity.com/paper/2407.01638/) (2024)
- [Reinforcement Learning and Adaptive Sampling for Optimized DNN Compilation](https://mlsanity.com/paper/1905.12799/) (2019)
- [Quantifying OpenMP: Statistical Insights into Usage and Adoption](https://mlsanity.com/paper/2308.08002/) (2023)
- [ComPar: Optimized Multi-Compiler for Automatic OpenMP S2S Parallelization](https://mlsanity.com/paper/2005.13304/) (2020)
- [LOOPer: A Learned Automatic Code Optimizer For Polyhedral Compilers](https://mlsanity.com/paper/2403.11522/) (2024)
- [Do Large Language Models Understand Performance Optimization?](https://mlsanity.com/paper/2503.13772/) (2025)
- [MPI-rical: Data-Driven MPI Distributed Parallelism Assistance with Transformers](https://mlsanity.com/paper/2305.09438/) (2023)
- [MPIrigen: MPI Code Generation through Domain-Specific Language Models](https://mlsanity.com/paper/2402.09126/) (2024)
- [OMPar: Automatic Parallelization with AI-Driven Source-to-Source Compilation](https://mlsanity.com/paper/2409.14771/) (2024)
- [Can Large Language Models Predict Parallel Code Performance?](https://mlsanity.com/paper/2505.03988/) (2025)
- [UniPar: A Unified LLM-Based Framework for Parallel and Accelerated Code Translation in HPC](https://mlsanity.com/paper/2509.12136/) (2025)

## Abstract (arXiv)

Parallel programming is central to HPC and AI, but producing code that is correct and fast remains challenging, especially for OpenMP GPU offload, where data movement and tuning dominate. Autonomous coding agents can compile, test, and profile on target hardware, but outputs are brittle without domain scaffolding. We present ParaCodex, an HPC-engineer workflow that turns a Codex-based agent into an autonomous OpenMP GPU offload system using staged hotspot analysis, explicit data planning, correctness gating, and profiling-guided refinement. We evaluate translation from serial CPU kernels to OpenMP GPU offload kernels on HeCBench, Rodinia, and NAS. After excluding five kernels, ParaCodex succeeded on all 31 valid kernels. The generated kernels improved GPU time over reference OpenMP implementations in 25/31 cases, achieving geometric-mean speedups of 3x on HeCBench and 5x on Rodinia, and outperforming a zero-shot Codex baseline on all suites. We also evaluate CUDA to OpenMP offload translation on ParEval, where ParaCodex maintains high compilation and validation rates in code-only and end-to-end settings.