mlsanityData to 7 Oct 2026

Comparing Llama-2 and GPT-3 LLMs for HPC kernels generation

arXiv 2309.07103 ↗First posted 12 Sep 2023cs.SE cs.AI cs.DCcited by 30

Not on the map. Why?

Abstract (arXiv)

We evaluate the use of the open-source Llama-2 model for generating well-known, high-performance computing kernels (e.g., AXPY, GEMV, GEMM) on different parallel programming models and languages (e.g., C++: OpenMP, OpenMP Offload, OpenACC, CUDA, HIP; Fortran: OpenMP, OpenMP Offload, OpenACC; Python: numpy, Numba, pyCUDA, cuPy; and Julia: Threads, CUDA.jl, AMDGPU.jl). We built upon our previous work that is based on the OpenAI Codex, which is a descendant of GPT-3, to generate similar kernels with simple prompts via GitHub Copilot. Our goal is to compare the accuracy of Llama-2 and our original GPT-3 baseline by using a similar metric. Llama-2 has a simplified model that shows competitive or even superior accuracy. We also report on the differences between these foundational large language models as generative AI continues to redefine human-computer interactions. Overall, Copilot generates codes that are more reliable but less optimized, whereas codes generated by Llama-2 are less reliable but more optimized when correct.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 7 of its 24 references that are in the map. First posted on arXiv 12 Sep 2023.

Builds on

7 of 24 references.

References · 7

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Evaluation of OpenAI Codex for HPC Parallel Programming Models Kernel Generationcited by 22
  4. 2023LM4HPC: Towards Effective Language Model Application in High-Performance Computingcited by 18
  5. 2017Effective Extensible Programming: Unleashing Julia on GPUscited by 16
  6. 2023HPC-Coder: Modeling Parallel Programs using Large Language Modelscited by 11
  7. 2023Evaluating performance and portability of high-level programming models: Julia, Python/Numba, and Kokkos on exascale nodescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 10; 2025: 8; 2026: 11.

30 papers.

Sort:

By area:

  1. 2025KernelBench: Can LLMs Write Efficient GPU Kernels?cited by 206
  2. 2024Can Large Language Models Write Parallel Code?cited by 36
  3. 2024The Landscape and Challenges of HPC Research and LLMscited by 22
  4. 2024Performance-Aligned LLMs for Generating Fast Codecited by 20
  5. 2024OMPGPT: A Generative Pre-trained Transformer Model for OpenMPcited by 16
  6. 2023LLM4VV: Developing LLM-Driven Testsuite for Compiler Validationcited by 15
  7. 2026KernelFoundry: Hardware-aware evolutionary GPU kernel optimizationcited by 11
  8. 2024Temporal Knowledge Graph Question Answering: A Surveycited by 9
  9. 2025LLM & HPC:Benchmarking DeepSeek's Performance in High-Performance Computing Taskscited by 8
  10. 2024LLM4GRN: Discovering Causal Gene Regulatory Networks with LLMs -- Evaluation through Synthetic Data Generationcited by 7
  11. 2026CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTecited by 6
  12. 2024MPIrigen: MPI Code Generation through Domain-Specific Language Modelscited by 5
18 more
  1. 2024OMPar: Automatic Parallelization with AI-Driven Source-to-Source Compilationcited by 5
  2. 2025Leveraging AI for Productive and Trustworthy HPC Software: Challenges and Research Directionscited by 2
  3. 2025VibeCodeHPC: An Agent-Based Iterative Prompting Auto-Tuner for HPC Code Generation Using LLMscited by 2
  4. 2024Should AI Optimize Your Code? A Comparative Study of Classical Optimizing Compilers Versus Current Large Language Modelscited by 1
  5. 2025Event Segmentation Applications in Large Language Model Enabled Automated Recall Assessmentscited by 1
  6. 2025AI Assistants to Enhance and Exploit the PETSc Knowledge Basecited by 1
  7. 2025Performance Evaluation of General Purpose Large Language Models for Basic Linear Algebra Subprograms Code Generationcited by 1
  8. 2026Exploring the Role of LLMs in HPC Programming: A Surveycited by 1
  9. 2024Automated Creation of Source Code Variants of a Cryptographic Hash Function Implementation Using Generative Pre-Trained Transformer Modelscited by 0
  10. 2025EPIC: Generative AI Platform for Accelerating HPC Operational Data Analyticscited by 0
  11. 2026ParaCodex: A Profiling-Guided Autonomous Coding Agent for Reliable Parallel Code Generation and Translationcited by 0
  12. 2026From Prompts to Performance: Evaluating LLMs for Task-based Parallel Code Generationcited by 0
  13. 2026An Agentic Evaluation Framework for AI-Generated Scientific Code in PETSccited by 0
  14. 2026AstraAI: LLMs, Retrieval, and AST-Guided Assistance for HPC Codebasescited by 0
  15. 2026Learning Reasoning World Models for Parallel Codecited by 0
  16. 2026CodegenBench: Can LLMs Write Efficient Code Across Architectures?cited by 0
  17. 2026Cross-Model Cross-Language AI Coding Agent Performance: Accuracy and Speed of Parallel CLRS Algorithmscited by 0
  18. 2026Effect of Abstractions and Prompting Strategies on LLM-Guided High-Performance Optimizationscited by 0