# Benchmarking Tabular Foundation Models as Surrogates in Expensive Evolutionary Optimization

arXiv: https://arxiv.org/abs/2609.18130 · HTML: https://mlsanity.com/paper/2609.18130/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 50 references that are in the map.
- First posted on arXiv 16 Sep 2026.

## Builds on (most cited first, 6 of 6)

- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://mlsanity.com/paper/2302.09419/) (2023)
- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [Evolution Gym: A Large-Scale Benchmark for Evolving Soft Robots](https://mlsanity.com/paper/2201.09863/) (2022)
- [A support vector regression-based multi-fidelity surrogate model](https://mlsanity.com/paper/1906.09439/) (2019)
- [Exploiting Function-Family Structure in Analog Circuit Optimization](https://mlsanity.com/paper/2512.00712/) (2025)

## Abstract (arXiv)

Surrogate-assisted evolutionary algorithms (SAEAs) are effective methods for solving expensive optimization problems (EOPs), where surrogate models replace most expensive evaluations and critically influence the final optimization results. In recent years, tabular foundation models have advanced rapidly, and the Tabular Prior-data Fitted Network (TabPFN) has been adopted as a surrogate model for EOPs due to its strong predictive capability, demonstrating promising performance. Motivated by its potential as a surrogate model in SAEAs, this work conducts a comprehensive study that combines extensive experiments with in-depth theoretical analysis to investigate the effectiveness of TabPFN. Specifically, we perform experiments across both offline and online SAEA settings, covering diverse problem scenarios such as single-objective, multi-objective, constrained, combinatorial, mixed-variable, and engineering optimization problems. In addition, we further analyze the advantages and limitations of TabPFN within SAEAs and provide practical guidelines for its application in different optimization settings. Results show that the effectiveness of TabPFN is highly problem dependent, and it cannot replace conventional surrogates universally. Overall, TabPFN should be adopted selectively according to data availability, landscape complexity, search space characteristics, and its role within the algorithm. Customized model management strategies and role-specific algorithm design are necessary to fully exploit its advantages and avoid its pitfalls.