# Uncertainty-Guided Model Selection for Tabular Foundation Models in Biomolecule Efficacy Prediction

arXiv: https://arxiv.org/abs/2510.02476 · HTML: https://mlsanity.com/paper/2510.02476/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 27 references that are in the map.
- First posted on arXiv 2 Oct 2025.

## Builds on (most cited first, 11 of 11)

- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [TabICL: A Tabular Foundation Model for In-Context Learning on Large Data](https://mlsanity.com/paper/2502.05564/) (2025)
- [Retrieval & Fine-Tuning for In-Context Tabular Models](https://mlsanity.com/paper/2406.05207/) (2024)
- [Selection and Aggregation of Conformal Prediction Sets](https://mlsanity.com/paper/2104.13871/) (2021)
- [Merging uncertainty sets via majority vote](https://mlsanity.com/paper/2401.09379/) (2024)
- [Conformal prediction after data-dependent model selection](https://mlsanity.com/paper/2408.07066/) (2024)
- [Towards Fair In-Context Learning with Tabular Foundation Models](https://mlsanity.com/paper/2505.09503/) (2025)
- [PSEO: Optimizing Post-hoc Stacking Ensemble Through Hyperparameter Tuning](https://mlsanity.com/paper/2508.05144/) (2025)
- [RRMSE Voting Regressor: A weighting function based improvement to ensemble regression](https://mlsanity.com/paper/2207.04837/) (2022)
- [Dynamic ensemble selection based on Deep Neural Network Uncertainty Estimation for Adversarial Robustness](https://mlsanity.com/paper/2308.00346/) (2023)
- [siDPT: siRNA Efficacy Prediction via Debiased Preference-Pair Transformer](https://mlsanity.com/paper/2509.15664/) (2025)

## Built on by (most cited first, 2 of 2)

- [TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models](https://mlsanity.com/paper/2511.08667/) (2025)
- [TabPFN-3: Technical Report](https://mlsanity.com/paper/2605.13986/) (2026)

## Abstract (arXiv)

In-context learners like TabPFN are promising for biomolecule efficacy prediction, where established molecular feature sets and relevant experimental results can serve as powerful contextual examples. However, their performance is highly sensitive to the provided context, making strategies like post-hoc ensembling of models trained on different data subsets a viable approach. An open question is how to select the best models for the ensemble without access to ground truth labels. In this study, we investigate an uncertainty-guided strategy for model selection. We demonstrate on an siRNA knockdown efficacy task that a TabPFN model using straightforward sequence-based features can surpass specialized state-of-the-art predictors. We also show that the model's predicted inter-quantile range (IQR), a measure of its uncertainty, has a negative correlation with true prediction error. We developed the OligoICP method, which selects and averages an ensemble of models with the lowest mean IQR for siRNA efficacy prediction, achieving superior performance compared to naive ensembling or using a single model trained on all available data. This finding highlights model uncertainty as a powerful, label-free heuristic for optimizing biomolecule efficacy predictions.