# Accelerating Latency-Critical Applications with AI-Powered Semi-Automatic Fine-Grained Parallelization on SMT Processors

arXiv: https://arxiv.org/abs/2509.00883 · HTML: https://mlsanity.com/paper/2509.00883/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 25 references that are in the map.
- First posted on arXiv 31 Aug 2025.

## Builds on (most cited first, 7 of 7)

- [Can Large Language Models Write Parallel Code?](https://mlsanity.com/paper/2401.12554/) (2024)
- [OMPGPT: A Generative Pre-trained Transformer Model for OpenMP](https://mlsanity.com/paper/2401.16445/) (2024)
- [Learning to Parallelize with OpenMP by Augmented Heterogeneous AST Representation](https://mlsanity.com/paper/2305.05779/) (2023)
- [MonoCoder: Domain-Specific Code Language Model for HPC Codes and Tasks](https://mlsanity.com/paper/2312.13322/) (2023)
- [Advising OpenMP Parallelization via a Graph-Based Approach with Transformers](https://mlsanity.com/paper/2305.11999/) (2023)
- [OMPar: Automatic Parallelization with AI-Driven Source-to-Source Compilation](https://mlsanity.com/paper/2409.14771/) (2024)
- [Exploring Fine-grained Task Parallelism on Simultaneous Multithreading Cores](https://mlsanity.com/paper/2410.01222/) (2024)

## Abstract (arXiv)

Latency-critical applications tend to show low utilization of functional units due to frequent cache misses and mispredictions during speculative execution in high-performance superscalar processors. However, due to significant impact on single-thread performance, Simultaneous Multithreading (SMT) technology is rarely used with heavy threads of latency-critical applications. In this paper, we explore utilization of SMT technology to support fine-grained parallelization of latency-critical applications. Following the advancements in the development of Large Language Models (LLMs), we introduce Aira, an AI-powered Parallelization Adviser. To implement Aira, we extend AI Coding Agent in Cursor IDE with additional tools connected through Model Context Protocol, enabling end-to-end AI Agent for parallelization. Additional connected tools enable LLM-guided hotspot detection, collection of dynamic dependencies with Dynamic Binary Instrumentation, SMT-aware performance simulation to estimate performance gains. We apply Aira with Relic parallel framework for fine-grained task parallelism on SMT cores to parallelize latency-critical benchmarks representing real-world applications used in industry. We show 17% geomean performance gain from parallelization of latency-critical benchmarks using Aira with Relic framework.