# Foundation-Model Surrogates Enable Data-Efficient Active Learning for Materials Discovery

arXiv: https://arxiv.org/abs/2603.12567 · HTML: https://mlsanity.com/paper/2603.12567/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 56 references that are in the map.
- First posted on arXiv 13 Mar 2026.

## Builds on (most cited first, 6 of 6)

- [Benchmarking the Performance of Bayesian Optimization across Multiple Experimental Materials Science Domains](https://mlsanity.com/paper/2106.01309/) (2021)
- [A Survey of Generative AI for de novo Drug Design: New Frontiers in Molecule and Protein Generation](https://mlsanity.com/paper/2402.08703/) (2024)
- [Bayesian Optimization in Materials Science: A Survey](https://mlsanity.com/paper/2108.00002/) (2021)
- [Accelerating material discovery with a threshold-driven hybrid acquisition policy-based Bayesian optimization](https://mlsanity.com/paper/2311.09591/) (2023)
- [Deep Gaussian Process-based Cost-Aware Batch Bayesian Optimization for Complex Materials Design Campaigns](https://mlsanity.com/paper/2509.14408/) (2025)
- [When Active Learning Fails, Uncalibrated Out of Distribution Uncertainty Quantification Might Be the Problem](https://mlsanity.com/paper/2511.17760/) (2025)

## Built on by (most cited first, 1 of 1)

- [TabPFN-3: Technical Report](https://mlsanity.com/paper/2605.13986/) (2026)

## Abstract (arXiv)

Active learning (AL) has emerged as a powerful paradigm for accelerating materials discovery by iteratively steering experiments toward promising candidates, reducing the number of costly synthesis-and-characterization cycles needed to identify optimal materials. However, current AL relies predominantly on Gaussian Process (GP) and Random Forest (RF) surrogates, which suffer from complementary limitations: GP underfits complex composition-property landscapes due to rigid kernel assumptions, while RF produces unreliable heuristic uncertainty estimates in small-data regimes. This small-data challenge is pervasive in materials science, making reliable surrogate modeling extremely difficult with models trained from scratch on each new dataset. Here we propose In-Context Active Learning (ICAL), which addresses this bottleneck by replacing conventional surrogates with TabPFN, a transformer-based foundation model (FM) pre-trained on millions of synthetic regression tasks to meta-learn a universal prior over tabular data, upon which TabPFN performs principled Bayesian inference in a single forward pass without dataset-specific retraining, delivering strong small-data regression performance and well-calibrated predictive uncertainty (required for effective AL). We benchmark ICAL against GP and RF across 10 materials datasets and TabPFN wins on 8 out of 10 datasets, achieving a mean saving of 52% in extra evaluations relative to GP and 29.77% relative to RF. Cross-validation analysis confirms that TabPFN's advantage stems from superior uncertainty calibration, achieving the lowest Negative Log-Likelihood and Area Under the Sparsification Error curve among all surrogates. These results demonstrate that pre-trained FMs can serve as effective surrogates for active learning, enabling data-efficient discovery across diverse materials systems and small-data experimental sciences.