# Attention-Informed Surrogates for Navigating Power-Performance Trade-offs in HPC

arXiv: https://arxiv.org/abs/2601.15399 · HTML: https://mlsanity.com/paper/2601.15399/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 21 Jan 2026.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2006.05078/) (2020)
- [Deep Reinforcement Agent for Scheduling in HPC](https://mlsanity.com/paper/2102.06243/) (2021)
- [HPC Storage Service Autotuning Using Variational-Autoencoder-Guided Asynchronous Bayesian Optimization](https://mlsanity.com/paper/2210.00798/) (2022)
- [Big Data Meets HPC Log Analytics: Scalable Approach to Understanding Systems at Extreme Scale](https://mlsanity.com/paper/1708.06884/) (2017)
- [Comparative Code Structure Analysis using Deep Learning for Performance Prediction](https://mlsanity.com/paper/2102.07660/) (2021)
- [DRAS-CQSim: A Reinforcement Learning based Framework for HPC Cluster Scheduling](https://mlsanity.com/paper/2105.07526/) (2021)
- [A Probabilistic Machine Learning Approach to Scheduling Parallel Loops with Bayesian Optimization](https://mlsanity.com/paper/2206.05787/) (2022)
- [A HPC Co-Scheduler with Reinforcement Learning](https://mlsanity.com/paper/2401.09706/) (2024)
- [Novel Representation Learning Technique using Graphs for Performance Analytics](https://mlsanity.com/paper/2401.10799/) (2024)
- [Scalable3-BO: Big Data meets HPC - A scalable asynchronous parallel high-dimensional Bayesian optimization framework on supercomputers](https://mlsanity.com/paper/2108.05969/) (2021)

## Abstract (arXiv)

High-Performance Computing (HPC) schedulers must balance user performance with facility-wide resource constraints. The task boils down to selecting the optimal number of nodes for a given job. We present a surrogate-assisted multi-objective Bayesian optimization (MOBO) framework to automate this complex decision. Our core hypothesis is that surrogate models informed by attention-based embeddings of job telemetry can capture performance dynamics more effectively than standard regression techniques. We pair this with an intelligent sample acquisition strategy to ensure the approach is data-efficient. On two production HPC datasets, our embedding-informed method consistently identified higher-quality Pareto fronts of runtime-power trade-offs compared to baselines. Furthermore, our intelligent data sampling strategy drastically reduced training costs while improving the stability of the results. To our knowledge, this is the first work to successfully apply embedding-informed surrogates in a MOBO framework to the HPC scheduling problem, jointly optimizing for performance and power on production workloads.