mlsanityData to 7 Oct 2026

Cross-Model Cross-Language AI Coding Agent Performance: Accuracy and Speed of Parallel CLRS Algorithms

arXiv 2607.26083 ↗First posted 26 Jul 2026cs.SEcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

AI coding agents have quickly become omnipresent in software engineering. Their serial performance, both in terms of accuracy and speed, has been extensively covered. However, recent initial results suggest their parallel programming capabilities lag behind serial programming capabilities. This paper presents a cross-language evaluation of three coding agents -- Cursor's Composer 2.0, GPT 5.4, and Claude Sonnet 4.6 -- on parallel code generation across three algorithm categories -- sorting, graph traversal, and search -- in C++, Python, and Julia. For each algorithm and language pair, we prompt a coding agent to produce a parallel implementation from a serial baseline, track the prompting effort required to achieve both functional correctness and performance improvements, and measure speedup against both custom serial baselines and third-party library implementations. We find that coding agents can produce correct parallel implementations with modest prompting effort, but that achieving meaningful speedup is heavily algorithm- and language-dependent. Sonnet 4.6 delivers the strongest overall performance gains, whereas GPT 5.4 produces no measurable speedups despite consistent correctness. C++ is most consistently parallelizable for graph algorithms, while Python and Julia achieve the largest speedups on search algorithms: no single language dominates across all categories. Python and Julia each achieve speedup on some graph algorithms but regress on others. These findings underscore the impact of including runtime performance efficiency as a main LLM performance metric, in addition to accuracy, particularly for parallel implementations.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 23 of its 28 references that are in the map. First posted on arXiv 26 Jul 2026.

Builds on

23 of 28 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities15 Aug
  2. 2025A Survey on Code Generation with LLM-based Agents31 Jul

Other references · 21

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2021Program Synthesis with Large Language Modelscited by 3,509
  3. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  4. 2025KernelBench: Can LLMs Write Efficient GPU Kernels?cited by 206
  5. 2024How Efficient is LLM-Generated Code? A Rigorous & High-Standard Benchmarkcited by 39
  6. 2024Can Large Language Models Write Parallel Code?cited by 36
  7. 2023Comparing Llama-2 and GPT-3 LLMs for HPC kernels generationcited by 30
  8. 2024The CLRS-Text Algorithmic Reasoning Language Benchmarkcited by 28
  9. 2024Performance-Aligned LLMs for Generating Fast Codecited by 20
  10. 2023LM4HPC: Towards Effective Language Model Application in High-Performance Computingcited by 18
  11. 2024OMPGPT: A Generative Pre-trained Transformer Model for OpenMPcited by 16
  12. 2026AutoKernel: Autonomous GPU Kernel Optimization via Iterative Agent-Driven Searchcited by 14
9 more
  1. 2023Scope is all you need: Transforming LLMs for HPC Codecited by 12
  2. 2026KernelFoundry: Hardware-aware evolutionary GPU kernel optimizationcited by 11
  3. 2024LASSI: An LLM-based Automated Self-Correcting Pipeline for Translating Parallel Scientific Codescited by 10
  4. 2025LLM & HPC:Benchmarking DeepSeek's Performance in High-Performance Computing Taskscited by 8
  5. 2025Do Large Language Models Understand Performance Optimization?cited by 6
  6. 2024Evaluating AI-generated code for C++, Fortran, Go, Java, Julia, Matlab, Python, R, and Rustcited by 5
  7. 2025PEAK: A Performance Engineering AI-Assistant for GPU Kernels Powered by Natural Language Transformationscited by 5
  8. 2026Fine-Tuning GPT-5 for GPU Kernel Generationcited by 4
  9. 2026HintPilot: LLM-based Compiler Hint Synthesis for Code Optimizationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026FormulaCode: Evaluating Agentic Optimization on Large Codebases
  2. 2026Frontier Coding Agents Use Metaprogramming to Adapt to Unfamiliar Programming Languages
  3. 2025SWE-Sharp-Bench: A Reproducible Benchmark for C# Software Engineering Tasks
  4. 2026The Best Programming Language for Tokenmaxxing: An Investigation of Coding Agent Behavior Across Programming Languages
  5. 2026Understanding the Architecture of Coding Agents: An Exploratory Study Using a Research Prototype
  6. 2026ProjDevBench: Benchmarking AI Coding Agents on End-to-End Project Development
  7. 2026EvoCode-Bench: Evaluating Coding Agents in Multi-Turn Iterative Interactions
  8. 2025MARCO: Multi-Agent Code Optimization with Real-Time Knowledge Integration for High-Performance Computing