mlsanityData to 7 Oct 2026

Performance Evaluation of General Purpose Large Language Models for Basic Linear Algebra Subprograms Code Generation

arXiv 2507.04697 ↗First posted 7 Jul 2025cs.LG cs.DC cs.MScited by 1

Not on the map. Why?

Abstract (arXiv)

Generative AI technology based on Large Language Models (LLM) has been developed and applied to assist or automatically generate program codes. In this paper, we evaluate the capability of existing general LLMs for Basic Linear Algebra Subprograms (BLAS) code generation for CPUs. We use two LLMs provided by OpenAI: GPT-4.1, a Generative Pre-trained Transformer (GPT) model, and o4-mini, one of the o-series of Reasoning models. Both have been released in April 2025. For the routines from level-1 to 3 BLAS, we tried to generate (1) C code without optimization from routine name only, (2) C code with basic performance optimizations (thread parallelization, SIMD vectorization, and cache blocking) from routine name only, and (3) C code with basic performance optimizations based on Fortran reference code. As a result, we found that correct code can be generated in many cases even when only routine name are given. We also confirmed that thread parallelization with OpenMP, SIMD vectorization, and cache blocking can be implemented to some extent, and that the code is faster than the reference code.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 17 of its 23 references that are in the map. First posted on arXiv 7 Jul 2025.

Builds on

17 of 23 references.

References · 17

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  6. 2023StarCoder: may the source be with you!cited by 790
  7. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 155
  8. 2024Can Large Language Models Write Parallel Code?cited by 36
  9. 2023Comparing Llama-2 and GPT-3 LLMs for HPC kernels generationcited by 30
  10. 2023Evaluation of OpenAI Codex for HPC Parallel Programming Models Kernel Generationcited by 22
  11. 2023HPC-GPT: Integrating Large Language Model for High-Performance Computingcited by 22
  12. 2023LM4HPC: Towards Effective Language Model Application in High-Performance Computingcited by 18
5 more
  1. 2025GraphCodeAgent: Dual Graph-Guided LLM Agent for Retrieval-Augmented Repo-Level Code Generationcited by 17
  2. 2024Enhancing Code Translation in Language Models with Few-Shot Learning via Retrieval-Augmented Generationcited by 16
  3. 2023MonoCoder: Domain-Specific Code Language Model for HPC Codes and Taskscited by 12
  4. 2025LLM-Assisted Translation of Legacy FORTRAN Codes to C++: A Cross-Platform Studycited by 7
  5. 2025ARCS: Agentic Retrieval-Augmented Code Synthesis with Iterative Refinementcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026CodegenBench: Can LLMs Write Efficient Code Across Architectures?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Holistic Evaluation of State-of-the-Art LLMs for Code Generation
  2. 2024Performance-Aligned LLMs for Generating Fast Code
  3. 2025Enhancing LLM Code Generation: A Systematic Evaluation of Multi-Agent Collaboration and Runtime Debugging for Improved Accuracy, Reliability, and Latency
  4. 2026Improving HPC Code Generation Capability of LLMs via Online Reinforcement Learning with Real-Machine Benchmark Rewards
  5. 2026Exploring the Role of LLMs in HPC Programming: A Survey
  6. 2024Steering Large Language Models between Code Execution and Textual Reasoning
  7. 2025Data-efficient LLM Fine-tuning for Code Generation
  8. 2025Exploring Code Language Models for Automated HLS-based Hardware Generation: Benchmark, Infrastructure and Analysis