# Learning to Parallelize with OpenMP by Augmented Heterogeneous AST Representation

arXiv: https://arxiv.org/abs/2305.05779 · HTML: https://mlsanity.com/paper/2305.05779/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 49 references that are in the map.
- First posted on arXiv 9 May 2023.

## Builds on (most cited first, 14 of 14)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Heterogeneous Graph Transformer](https://mlsanity.com/paper/2003.01332/) (2020)
- [Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/1809.05679/) (2018)
- [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018)
- [Learning to Represent Programs with Graphs](https://mlsanity.com/paper/1711.00740/) (2017)
- [Language-Agnostic Representation Learning of Source Code from Structure and Context](https://mlsanity.com/paper/2103.11318/) (2021)
- [Text Level Graph Neural Network for Text Classification](https://mlsanity.com/paper/1910.02356/) (2019)
- [Neural Code Comprehension: A Learnable Representation of Code Semantics](https://mlsanity.com/paper/1806.07336/) (2018)
- [Image Classification using Graph Neural Network and Multiscale Wavelet Superpixels](https://mlsanity.com/paper/2201.12633/) (2022)
- [Learning to Parallelize in a Shared-Memory Environment with Transformers](https://mlsanity.com/paper/2204.12835/) (2022)
- [Learning Intermediate Representations using Graph Neural Networks for NUMA and Prefetchers Optimization](https://mlsanity.com/paper/2203.00611/) (2022)
- [MolNet: A Chemically Intuitive Graph Neural Network for Prediction of Molecular Properties](https://mlsanity.com/paper/2203.09456/) (2022)

## Built on by (most cited first, 14 of 14)

- [HPC-GPT: Integrating Large Language Model for High-Performance Computing](https://mlsanity.com/paper/2311.12833/) (2023)
- [The Landscape and Challenges of HPC Research and LLMs](https://mlsanity.com/paper/2402.02018/) (2024)
- [LM4HPC: Towards Effective Language Model Application in High-Performance Computing](https://mlsanity.com/paper/2306.14979/) (2023)
- [OMPGPT: A Generative Pre-trained Transformer Model for OpenMP](https://mlsanity.com/paper/2401.16445/) (2024)
- [Scope is all you need: Transforming LLMs for HPC Code](https://mlsanity.com/paper/2308.09440/) (2023)
- [MonoCoder: Domain-Specific Code Language Model for HPC Codes and Tasks](https://mlsanity.com/paper/2312.13322/) (2023)
- [Data Race Detection Using Large Language Models](https://mlsanity.com/paper/2308.07505/) (2023)
- [AutoParLLM: GNN-guided Context Generation for Zero-Shot Code Parallelization using LLMs](https://mlsanity.com/paper/2310.04047/) (2023)
- [PERFOGRAPH: A Numerical Aware Program Graph Representation for Performance Optimization and Program Analysis](https://mlsanity.com/paper/2306.00210/) (2023)
- [Beyond Code Pairs: Dialogue-Based Data Generation for LLM Code Translation](https://mlsanity.com/paper/2512.03086/) (2025)
- [MIREncoder: Multi-modal IR-based Pretrained Embeddings for Performance Optimizations](https://mlsanity.com/paper/2407.02238/) (2024)
- [Accelerating Latency-Critical Applications with AI-Powered Semi-Automatic Fine-Grained Parallelization on SMT Processors](https://mlsanity.com/paper/2509.00883/) (2025)
- [ACCeLLiuM: Supervised Fine-Tuning for Automated OpenACC Pragma Generation](https://mlsanity.com/paper/2509.20380/) (2025)
- [ParaCodex: A Profiling-Guided Autonomous Coding Agent for Reliable Parallel Code Generation and Translation](https://mlsanity.com/paper/2601.04327/) (2026)

## Abstract (arXiv)

Detecting parallelizable code regions is a challenging task, even for experienced developers. Numerous recent studies have explored the use of machine learning for code analysis and program synthesis, including parallelization, in light of the success of machine learning in natural language processing. However, applying machine learning techniques to parallelism detection presents several challenges, such as the lack of an adequate dataset for training, an effective code representation with rich information, and a suitable machine learning model to learn the latent features of code for diverse analyses. To address these challenges, we propose a novel graph-based learning approach called Graph2Par that utilizes a heterogeneous augmented abstract syntax tree (Augmented-AST) representation for code. The proposed approach primarily focused on loop-level parallelization with OpenMP. Moreover, we create an OMPSerial dataset with 18598 parallelizable and 13972 non-parallelizable loops to train the machine learning models. Our results show that our proposed approach achieves the accuracy of parallelizable code region detection with 85% accuracy and outperforms the state-of-the-art token-based machine learning approach. These results indicate that our approach is competitive with state-of-the-art tools and capable of handling loops with complex structures that other tools may overlook.