# LLM & HPC:Benchmarking DeepSeek's Performance in High-Performance Computing Tasks

arXiv: https://arxiv.org/abs/2504.03665 · HTML: https://mlsanity.com/paper/2504.03665/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 15 Mar 2025.

## Builds on (most cited first, 13 of 13)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [Comparing Llama-2 and GPT-3 LLMs for HPC kernels generation](https://mlsanity.com/paper/2309.07103/) (2023)
- [Evaluation of OpenAI Codex for HPC Parallel Programming Models Kernel Generation](https://mlsanity.com/paper/2306.15121/) (2023)
- [HPC-GPT: Integrating Large Language Model for High-Performance Computing](https://mlsanity.com/paper/2311.12833/) (2023)
- [The Landscape and Challenges of HPC Research and LLMs](https://mlsanity.com/paper/2402.02018/) (2024)
- [LM4HPC: Towards Effective Language Model Application in High-Performance Computing](https://mlsanity.com/paper/2306.14979/) (2023)
- [LLM4VV: Developing LLM-Driven Testsuite for Compiler Validation](https://mlsanity.com/paper/2310.04963/) (2023)
- [Scope is all you need: Transforming LLMs for HPC Code](https://mlsanity.com/paper/2308.09440/) (2023)
- [Creating a Dataset for High-Performance Computing Code Translation using LLMs: A Bridge Between OpenMP Fortran and C++](https://mlsanity.com/paper/2307.07686/) (2023)
- [Evaluating AI-generated code for C++, Fortran, Go, Java, Julia, Matlab, Python, R, and Rust](https://mlsanity.com/paper/2405.13101/) (2024)
- [Benchmarking the Parallel 1D Heat Equation Solver in Chapel, Charm++, C++, HPX, Go, Julia, Python, Rust, Swift, and Java](https://mlsanity.com/paper/2307.01117/) (2023)

## Built on by (most cited first, 8 of 8)

- [LLM Benchmarking with LLaMA2: Evaluating Code Development Performance Across Multiple Programming Languages](https://mlsanity.com/paper/2503.19217/) (2025)
- [Can LLMs Find Bugs in Code? An Evaluation from Beginner Errors to Security Vulnerabilities in Python and C++](https://mlsanity.com/paper/2508.16419/) (2025)
- [An Agentic Evaluation Framework for AI-Generated Scientific Code in PETSc](https://mlsanity.com/paper/2603.15976/) (2026)
- [GTBench: A Curriculum-Grounded Benchmark for Evaluating LLMs as Mathematical Research Assistants in Graph Theory](https://mlsanity.com/paper/2606.03144/) (2026)
- [AstraAI: LLMs, Retrieval, and AST-Guided Assistance for HPC Codebases](https://mlsanity.com/paper/2603.27423/) (2026)
- [Learning Reasoning World Models for Parallel Code](https://mlsanity.com/paper/2604.20926/) (2026)
- [Cross-Model Cross-Language AI Coding Agent Performance: Accuracy and Speed of Parallel CLRS Algorithms](https://mlsanity.com/paper/2607.26083/) (2026)
- [HARGO: Heterogeneity-Aware Reward-Guided Optimization for RL Post-Training of LLMs on HPC Tasks](https://mlsanity.com/paper/2607.28301/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs), such as GPT-4 and DeepSeek, have been applied to a wide range of domains in software engineering. However, their potential in the context of High-Performance Computing (HPC) much remains to be explored. This paper evaluates how well DeepSeek, a recent LLM, performs in generating a set of HPC benchmark codes: a conjugate gradient solver, the parallel heat equation, parallel matrix multiplication, DGEMM, and the STREAM triad operation. We analyze DeepSeek's code generation capabilities for traditional HPC languages like Cpp, Fortran, Julia and Python. The evaluation includes testing for code correctness, performance, and scaling across different configurations and matrix sizes. We also provide a detailed comparison between DeepSeek and another widely used tool: GPT-4. Our results demonstrate that while DeepSeek generates functional code for HPC tasks, it lags behind GPT-4, in terms of scalability and execution efficiency of the generated code.