# Multi-objective Optimization in CPU Design Space Exploration: Attention is All You Need

arXiv: https://arxiv.org/abs/2410.18368 · HTML: https://mlsanity.com/paper/2410.18368/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 49 references that are in the map.
- First posted on arXiv 24 Oct 2024.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Regression Transformer: Concurrent sequence regression and generation for molecular language modeling](https://mlsanity.com/paper/2202.01338/) (2022)
- [SCAR: Spatial-/Channel-wise Attention Regression Networks for Crowd Counting](https://mlsanity.com/paper/1908.03716/) (2019)
- [Projection Pursuit Gaussian Process Regression](https://mlsanity.com/paper/2004.00667/) (2020)
- [Learning Pareto-Frontier Resource Management Policies for Heterogeneous SoCs: An Information-Theoretic Approach](https://mlsanity.com/paper/2105.09282/) (2021)

## Built on by (most cited first, 1 of 1)

- [MetaDSE: A Few-shot Meta-learning Framework for Cross-workload CPU Design Space Exploration](https://mlsanity.com/paper/2504.13568/) (2025)

## Abstract (arXiv)

Design Space Exploration (DSE) is essential to modern CPU design, yet current frameworks struggle to scale and generalize in high-dimensional architectural spaces. As the dimensionality of design spaces continues to grow, existing DSE frameworks face three fundamental challenges: (1) reduced accuracy and poor scalability of surrogate models in large design spaces; (2) inefficient acquisition guided by hand-crafted heuristics or exhaustive search; (3) limited interpretability, making it hard to pinpoint architectural bottlenecks. In this work, we present AttentionDSE, the first end-to-end DSE framework that natively integrates performance prediction and design guidance through an attention-based neural architecture. Unlike traditional DSE workflows that separate surrogate modeling from acquisition and rely heavily on hand-crafted heuristics, AttentionDSE establishes a unified, learning-driven optimization loop, in which attention weights serve a dual role: enabling accurate performance estimation and simultaneously exposing the performance bottleneck. This paradigm shift elevates attention from a passive representation mechanism to an active, interpretable driver of design decision-making. Key innovations include: (1) a Perception-Driven Attention mechanism that exploits architectural hierarchy and locality, scaling attention complexity from O(n2) to O(n) via sliding windows; (2) an Attention-aware Bottleneck Analysis that automatically surfaces critical parameters for targeted optimization, eliminating the need for domain-specific heuristics. Evaluated on high-dimensional CPU design space using the SPEC CPU2017 benchmark suite, AttentionDSE achieves up to 3.9% higher Pareto Hypervolume and over 80% reduction in exploration time compared to state-of-the-art baselines.