# Cross-Model Cross-Language AI Coding Agent Performance: Accuracy and Speed of Parallel CLRS Algorithms

arXiv: https://arxiv.org/abs/2607.26083 · HTML: https://mlsanity.com/paper/2607.26083/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 23 of its 28 references that are in the map.
- First posted on arXiv 26 Jul 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 23 of 23)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation](https://mlsanity.com/paper/2211.11501/) (2022)
- [KernelBench: Can LLMs Write Efficient GPU Kernels?](https://mlsanity.com/paper/2502.10517/) (2025)
- [A Survey on Code Generation with LLM-based Agents](https://mlsanity.com/paper/2508.00083/) (2025)
- [How Efficient is LLM-Generated Code? A Rigorous & High-Standard Benchmark](https://mlsanity.com/paper/2406.06647/) (2024)
- [Can Large Language Models Write Parallel Code?](https://mlsanity.com/paper/2401.12554/) (2024)
- [Comparing Llama-2 and GPT-3 LLMs for HPC kernels generation](https://mlsanity.com/paper/2309.07103/) (2023)
- [The CLRS-Text Algorithmic Reasoning Language Benchmark](https://mlsanity.com/paper/2406.04229/) (2024)
- [AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities](https://mlsanity.com/paper/2508.11126/) (2025)
- [Performance-Aligned LLMs for Generating Fast Code](https://mlsanity.com/paper/2404.18864/) (2024)
- [LM4HPC: Towards Effective Language Model Application in High-Performance Computing](https://mlsanity.com/paper/2306.14979/) (2023)
- [AutoKernel: Autonomous GPU Kernel Optimization via Iterative Agent-Driven Search](https://mlsanity.com/paper/2603.21331/) (2026)
- [OMPGPT: A Generative Pre-trained Transformer Model for OpenMP](https://mlsanity.com/paper/2401.16445/) (2024)
- [KernelFoundry: Hardware-aware evolutionary GPU kernel optimization](https://mlsanity.com/paper/2603.12440/) (2026)
- [Scope is all you need: Transforming LLMs for HPC Code](https://mlsanity.com/paper/2308.09440/) (2023)
- [LASSI: An LLM-based Automated Self-Correcting Pipeline for Translating Parallel Scientific Codes](https://mlsanity.com/paper/2407.01638/) (2024)
- [LLM & HPC:Benchmarking DeepSeek's Performance in High-Performance Computing Tasks](https://mlsanity.com/paper/2504.03665/) (2025)
- [Do Large Language Models Understand Performance Optimization?](https://mlsanity.com/paper/2503.13772/) (2025)
- [Evaluating AI-generated code for C++, Fortran, Go, Java, Julia, Matlab, Python, R, and Rust](https://mlsanity.com/paper/2405.13101/) (2024)
- [PEAK: A Performance Engineering AI-Assistant for GPU Kernels Powered by Natural Language Transformations](https://mlsanity.com/paper/2512.19018/) (2025)
- [Fine-Tuning GPT-5 for GPU Kernel Generation](https://mlsanity.com/paper/2602.11000/) (2026)
- [HintPilot: LLM-based Compiler Hint Synthesis for Code Optimization](https://mlsanity.com/paper/2604.15041/) (2026)

## Abstract (arXiv)

AI coding agents have quickly become omnipresent in software engineering. Their serial performance, both in terms of accuracy and speed, has been extensively covered. However, recent initial results suggest their parallel programming capabilities lag behind serial programming capabilities. This paper presents a cross-language evaluation of three coding agents -- Cursor's Composer 2.0, GPT 5.4, and Claude Sonnet 4.6 -- on parallel code generation across three algorithm categories -- sorting, graph traversal, and search -- in C++, Python, and Julia. For each algorithm and language pair, we prompt a coding agent to produce a parallel implementation from a serial baseline, track the prompting effort required to achieve both functional correctness and performance improvements, and measure speedup against both custom serial baselines and third-party library implementations. We find that coding agents can produce correct parallel implementations with modest prompting effort, but that achieving meaningful speedup is heavily algorithm- and language-dependent. Sonnet 4.6 delivers the strongest overall performance gains, whereas GPT 5.4 produces no measurable speedups despite consistent correctness. C++ is most consistently parallelizable for graph algorithms, while Python and Julia achieve the largest speedups on search algorithms: no single language dominates across all categories. Python and Julia each achieve speedup on some graph algorithms but regress on others. These findings underscore the impact of including runtime performance efficiency as a main LLM performance metric, in addition to accuracy, particularly for parallel implementations.