# TransOpt: Transformer-based Representation Learning for Optimization Problem Classification

arXiv: https://arxiv.org/abs/2311.18035 · HTML: https://mlsanity.com/paper/2311.18035/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 12 references that are in the map.
- First posted on arXiv 29 Nov 2023.

## Builds on (most cited first, 6 of 6)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting](https://mlsanity.com/paper/1603.08785/) (2016)
- [Exploratory Landscape Analysis is Strongly Sensitive to the Sampling Strategy](https://mlsanity.com/paper/2006.11135/) (2020)
- [DoE2Vec: Deep-learning Based Features for Exploratory Landscape Analysis](https://mlsanity.com/paper/2304.01219/) (2023)
- [SELECTOR: Selecting a Representative Benchmark Suite for Reproducible Statistical Comparison](https://mlsanity.com/paper/2204.11527/) (2022)
- [A Collection of Deep Learning-based Feature-Free Approaches for Characterizing Single-Objective Continuous Fitness Landscapes](https://mlsanity.com/paper/2204.05752/) (2022)

## Built on by (most cited first, 3 of 3)

- [Deep-ELA: Deep Exploratory Landscape Analysis with Self-Supervised Pretrained Transformers for Single- and Multi-Objective Continuous Optimization Problems](https://mlsanity.com/paper/2401.01192/) (2024)
- [Landscape Features in Single-Objective Continuous Optimization: Have We Hit a Wall in Algorithm Selection Generalization?](https://mlsanity.com/paper/2501.17663/) (2025)
- [Evaluating Real-World Generalizability of Algorithm Selection Models](https://mlsanity.com/paper/2606.02016/) (2026)

## Abstract (arXiv)

We propose a representation of optimization problem instances using a transformer-based neural network architecture trained for the task of problem classification of the 24 problem classes from the Black-box Optimization Benchmarking (BBOB) benchmark. We show that transformer-based methods can be trained to recognize problem classes with accuracies in the range of 70%-80% for different problem dimensions, suggesting the possible application of transformer architectures in acquiring representations for black-box optimization problems.