# MIREncoder: Multi-modal IR-based Pretrained Embeddings for Performance Optimizations

arXiv: https://arxiv.org/abs/2407.02238 · HTML: https://mlsanity.com/paper/2407.02238/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 60 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Builds on (most cited first, 22 of 22)

- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [VideoBERT: A Joint Model for Video and Language Representation Learning](https://mlsanity.com/paper/1904.01766/) (2019)
- [Multimodal Deep Learning](https://mlsanity.com/paper/2301.04856/) (2023)
- [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018)
- [A Survey of Machine Learning for Big Code and Naturalness](https://mlsanity.com/paper/1709.06182/) (2017)
- [Learning to Represent Programs with Graphs](https://mlsanity.com/paper/1711.00740/) (2017)
- [Graph Matching Networks for Learning the Similarity of Graph Structured Objects](https://mlsanity.com/paper/1904.12787/) (2019)
- [CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks](https://mlsanity.com/paper/2105.12655/) (2021)
- [Neural Code Comprehension: A Learnable Representation of Code Semantics](https://mlsanity.com/paper/1806.07336/) (2018)
- [Large Language Models for Compiler Optimization](https://mlsanity.com/paper/2309.07062/) (2023)
- [NeuroVectorizer: End-to-End Vectorization with Deep Reinforcement Learning](https://mlsanity.com/paper/1909.13639/) (2019)
- [A General Path-Based Representation for Predicting Program Properties](https://mlsanity.com/paper/1803.09544/) (2018)
- [Learning to Parallelize with OpenMP by Augmented Heterogeneous AST Representation](https://mlsanity.com/paper/2305.05779/) (2023)
- [Autotuning PolyBench Benchmarks with LLVM Clang/Polly Loop Optimization Pragmas Using Bayesian Optimization](https://mlsanity.com/paper/2010.08040/) (2020)
- [ytopt: Autotuning Scientific Applications for Energy Efficiency at Large Scales](https://mlsanity.com/paper/2303.16245/) (2023)
- [Learning Intermediate Representations using Graph Neural Networks for NUMA and Prefetchers Optimization](https://mlsanity.com/paper/2203.00611/) (2022)
- [Quantifying OpenMP: Statistical Insights into Usage and Adoption](https://mlsanity.com/paper/2308.08002/) (2023)
- [A deep tree-based model for software defect prediction](https://mlsanity.com/paper/1802.00921/) (2018)
- [Performance Optimization using Multimodal Modeling and Heterogeneous GNN](https://mlsanity.com/paper/2304.12568/) (2023)
- [LS-CAT: A Large-Scale CUDA AutoTuning Dataset](https://mlsanity.com/paper/2103.14409/) (2021)

## Built on by (most cited first, 2 of 2)

- [LLM-Based Embeddings for Program Analysis and Optimization](https://mlsanity.com/paper/2608.07894/) (2026)
- [Accelerating Transfer-Learning-Based Autotuning with Predictive LLVM IR Performance Ranking](https://mlsanity.com/paper/2609.15807/) (2026)

## Abstract (arXiv)

One of the primary areas of interest in High Performance Computing is the improvement of performance of parallel workloads. Nowadays, compilable source code-based optimization tasks that employ deep learning often exploit LLVM Intermediate Representations (IRs) for extracting features from source code. Most such works target specific tasks, or are designed with a pre-defined set of heuristics. So far, pre-trained models are rare in this domain, but the possibilities have been widely discussed. Especially approaches mimicking large-language models (LLMs) have been proposed. But these have prohibitively large training costs. In this paper, we propose MIREncoder, a M}ulti-modal IR-based Auto-Encoder that can be pre-trained to generate a learned embedding space to be used for downstream tasks by machine learning-based approaches. A multi-modal approach enables us to better extract features from compilable programs. It allows us to better model code syntax, semantics and structure. For code-based performance optimizations, these features are very important while making optimization decisions. A pre-trained model/embedding implicitly enables the usage of transfer learning, and helps move away from task-specific trained models. Additionally, a pre-trained model used for downstream performance optimization should itself have reduced overhead, and be easily usable. These considerations have led us to propose a modeling approach that i) understands code semantics and structure, ii) enables use of transfer learning, and iii) is small and simple enough to be easily re-purposed or reused even with low resource availability. Our evaluations will show that our proposed approach can outperform the state of the art while reducing overhead.