mlsanityData to 7 Oct 2026

LM4HPC: Towards Effective Language Model Application in High-Performance Computing

arXiv 2306.14979 ↗First posted 26 Jun 2023cs.LG cs.DCcited by 18

Not on the map. Why?

Abstract (arXiv)

In recent years, language models (LMs), such as GPT-4, have been widely used in multiple domains, including natural language processing, visualization, and so on. However, applying them for analyzing and optimizing high-performance computing (HPC) software is still challenging due to the lack of HPC-specific support. In this paper, we design the LM4HPC framework to facilitate the research and development of HPC software analyses and optimizations using LMs. Tailored for supporting HPC datasets, AI models, and pipelines, our framework is built on top of a range of components from different levels of the machine learning software stack, with Hugging Face-compatible APIs. Using three representative tasks, we evaluated the prototype of our framework. The results show that LM4HPC can help users quickly evaluate a set of state-of-the-art models and generate insightful leaderboards.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 16 of its 27 references that are in the map. First posted on arXiv 26 Jun 2023.

Builds on

16 of 27 references.

References · 16

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  7. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  8. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  9. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  10. 2023StarCoder: may the source be with you!cited by 790
  11. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 468
  12. 2023LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructionscited by 124
4 more
  1. 2023Learning to Parallelize with OpenMP by Augmented Heterogeneous AST Representationcited by 14
  2. 2022Learning to Parallelize in a Shared-Memory Environment with Transformerscited by 10
  3. 2022Finding Reusable Machine Learning Components to Build Programming Language Processing Pipelinescited by 3
  4. 2022Towards Seamless Management of AI Models in High-Performance Computingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 4; 2025: 6; 2026: 2.

18 papers.

Sort:

By area:

  1. 2023Comparing Llama-2 and GPT-3 LLMs for HPC kernels generationcited by 30
  2. 2023HPC-GPT: Integrating Large Language Model for High-Performance Computingcited by 22
  3. 2024The Landscape and Challenges of HPC Research and LLMscited by 22
  4. 2024Performance-Aligned LLMs for Generating Fast Codecited by 20
  5. 2024OMPGPT: A Generative Pre-trained Transformer Model for OpenMPcited by 16
  6. 2023Can Large Language Models Find And Fix Vulnerable Software?cited by 14
  7. 2023Scope is all you need: Transforming LLMs for HPC Codecited by 12
  8. 2023MonoCoder: Domain-Specific Code Language Model for HPC Codes and Taskscited by 12
  9. 2023Data Race Detection Using Large Language Modelscited by 11
  10. 2024ChatVis: Automating Scientific Visualization with a Large Language Modelcited by 11
  11. 2025LLM & HPC:Benchmarking DeepSeek's Performance in High-Performance Computing Taskscited by 8
  12. 2025VizGenie: Toward Self-Refining, Domain-Aware Workflows for Next-Generation Scientific Visualizationcited by 8
6 more
  1. 2025ChatVis: Large Language Model Agent for Generating Scientific Visualizationscited by 7
  2. 2025MARCO: Multi-Agent Code Optimization with Real-Time Knowledge Integration for High-Performance Computingcited by 5
  3. 2025AI Assistants to Enhance and Exploit the PETSc Knowledge Basecited by 1
  4. 2025Performance Evaluation of General Purpose Large Language Models for Basic Linear Algebra Subprograms Code Generationcited by 1
  5. 2026AstraAI: LLMs, Retrieval, and AST-Guided Assistance for HPC Codebasescited by 0
  6. 2026Cross-Model Cross-Language AI Coding Agent Performance: Accuracy and Speed of Parallel CLRS Algorithmscited by 0