# Performance Evaluation of General Purpose Large Language Models for Basic Linear Algebra Subprograms Code Generation

arXiv: https://arxiv.org/abs/2507.04697 · HTML: https://mlsanity.com/paper/2507.04697/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 7 Jul 2025.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning](https://mlsanity.com/paper/2310.03731/) (2023)
- [Can Large Language Models Write Parallel Code?](https://mlsanity.com/paper/2401.12554/) (2024)
- [Comparing Llama-2 and GPT-3 LLMs for HPC kernels generation](https://mlsanity.com/paper/2309.07103/) (2023)
- [Evaluation of OpenAI Codex for HPC Parallel Programming Models Kernel Generation](https://mlsanity.com/paper/2306.15121/) (2023)
- [HPC-GPT: Integrating Large Language Model for High-Performance Computing](https://mlsanity.com/paper/2311.12833/) (2023)
- [LM4HPC: Towards Effective Language Model Application in High-Performance Computing](https://mlsanity.com/paper/2306.14979/) (2023)
- [GraphCodeAgent: Dual Graph-Guided LLM Agent for Retrieval-Augmented Repo-Level Code Generation](https://mlsanity.com/paper/2504.10046/) (2025)
- [Enhancing Code Translation in Language Models with Few-Shot Learning via Retrieval-Augmented Generation](https://mlsanity.com/paper/2407.19619/) (2024)
- [MonoCoder: Domain-Specific Code Language Model for HPC Codes and Tasks](https://mlsanity.com/paper/2312.13322/) (2023)
- [LLM-Assisted Translation of Legacy FORTRAN Codes to C++: A Cross-Platform Study](https://mlsanity.com/paper/2504.15424/) (2025)
- [ARCS: Agentic Retrieval-Augmented Code Synthesis with Iterative Refinement](https://mlsanity.com/paper/2504.20434/) (2025)

## Built on by (most cited first, 1 of 1)

- [CodegenBench: Can LLMs Write Efficient Code Across Architectures?](https://mlsanity.com/paper/2606.04023/) (2026)

## Abstract (arXiv)

Generative AI technology based on Large Language Models (LLM) has been developed and applied to assist or automatically generate program codes. In this paper, we evaluate the capability of existing general LLMs for Basic Linear Algebra Subprograms (BLAS) code generation for CPUs. We use two LLMs provided by OpenAI: GPT-4.1, a Generative Pre-trained Transformer (GPT) model, and o4-mini, one of the o-series of Reasoning models. Both have been released in April 2025. For the routines from level-1 to 3 BLAS, we tried to generate (1) C code without optimization from routine name only, (2) C code with basic performance optimizations (thread parallelization, SIMD vectorization, and cache blocking) from routine name only, and (3) C code with basic performance optimizations based on Fortran reference code. As a result, we found that correct code can be generated in many cases even when only routine name are given. We also confirmed that thread parallelization with OpenMP, SIMD vectorization, and cache blocking can be implemented to some extent, and that the code is faster than the reference code.