# Tabular foundation models for in-context prediction of molecular properties

arXiv: https://arxiv.org/abs/2604.16123 · HTML: https://mlsanity.com/paper/2604.16123/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 44 references that are in the map.
- First posted on arXiv 17 Apr 2026.

## Builds on (most cited first, 22 of 22)

- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data](https://mlsanity.com/paper/2003.06505/) (2020)
- [TabICL: A Tabular Foundation Model for In-Context Learning on Large Data](https://mlsanity.com/paper/2502.05564/) (2025)
- [Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development](https://mlsanity.com/paper/2102.09548/) (2021)
- [Large-Scale Chemical Language Representations Capture Molecular Structure and Properties](https://mlsanity.com/paper/2106.09553/) (2021)
- [TabArena: A Living Benchmark for Machine Learning on Tabular Data](https://mlsanity.com/paper/2506.16791/) (2025)
- [TabICLv2: A better, faster, scalable, and open tabular foundation model](https://mlsanity.com/paper/2602.11139/) (2026)
- [TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models](https://mlsanity.com/paper/2511.08667/) (2025)
- [Low Data Drug Discovery with One-shot Learning](https://mlsanity.com/paper/1611.03199/) (2016)
- [polyBERT: A chemical language model to enable fully machine-driven ultrafast polymer informatics](https://mlsanity.com/paper/2209.14803/) (2022)
- [Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Language](https://mlsanity.com/paper/2303.03363/) (2023)
- [TransPolymer: a Transformer-based language model for polymer property predictions](https://mlsanity.com/paper/2209.01307/) (2022)
- [Real-TabPFN: Improving Tabular Foundation Models via Continued Pre-training With Real-World Data](https://mlsanity.com/paper/2507.03971/) (2025)
- [textttMiniMol: A Parameter-Efficient Foundation Model for Molecular Learning](https://mlsanity.com/paper/2404.14986/) (2024)
- [Benchmarking Pretrained Molecular Embedding Models For Molecular Representation Learning](https://mlsanity.com/paper/2508.06199/) (2025)
- [Deep Learning Foundation Models from Classical Molecular Descriptors](https://mlsanity.com/paper/2506.15792/) (2025)
- [Gibbs-Helmholtz Graph Neural Network: capturing the temperature dependency of activity coefficients at infinite dilution](https://mlsanity.com/paper/2212.01199/) (2022)
- [PolyCL: Contrastive Learning for Polymer Representation Learning via Explicit and Implicit Augmentations](https://mlsanity.com/paper/2408.07556/) (2024)
- [Be aware of overfitting by hyperparameter optimization!](https://mlsanity.com/paper/2407.20786/) (2024)
- [Molecular Machine Learning in Chemical Process Design](https://mlsanity.com/paper/2508.20527/) (2025)
- [Superior Molecular Representations from Intermediate Encoder Layers](https://mlsanity.com/paper/2506.06443/) (2025)
- [Multitask-Informed Prior for In-Context Learning on Tabular Data: Application to Steel Property Prediction](https://mlsanity.com/paper/2603.22738/) (2026)

## Built on by (most cited first, 6 of 6)

- [TabPFN-3: Technical Report](https://mlsanity.com/paper/2605.13986/) (2026)
- [When Tabular Foundation Models Transfer Across Modalities: A Systematic Evaluation Across 95 Datasets, 7 Modalities, and Two Regimes](https://mlsanity.com/paper/2606.02106/) (2026)
- [Feature Selection for Tabular Foundation Models: Exploring Feature Utility in Low-Data Molecular Prediction](https://mlsanity.com/paper/2609.30789/) (2026)
- [Message Passing Does More with Less for In-Context Learning on Graphs](https://mlsanity.com/paper/2609.37057/) (2026)
- [Molecular Property Prediction under Structural Shift with Tabular Foundation Models](https://mlsanity.com/paper/2609.38744/) (2026)
- [Few-Shot Bioactivity Prediction with Meta-Learning under Assay Heterogeneity](https://mlsanity.com/paper/2610.07079/) (2026)

## Abstract (arXiv)

Accurate molecular property prediction is central to drug discovery, catalysis, and process design, yet real-world applications are often limited by small datasets. Molecular foundation models provide a promising direction by learning transferable molecular representations; however, they typically involve task-specific fine-tuning, require machine learning expertise, and often fail to outperform classical baselines. Tabular foundation models (TFMs) offer a fundamentally different paradigm: they perform predictions through in-context learning, enabling inference without task-specific training. Here, we evaluate TFMs in the low- to medium-data regime across both standardized pharmaceutical benchmarks and chemical engineering datasets. We evaluate both frozen molecular foundation model representations, as well as classical descriptors and fingerprints. Across the benchmarks, the approach shows excellent predictive performance while reducing computational cost, compared to fine-tuning, with these advantages also transferring to practical engineering data settings. In particular, combining TFMs with CheMeleon embeddings yields up to 100% win rates on 30 MoleculeACE tasks, while compact RDKit2d and Mordred descriptors provide strong descriptor-based alternatives. Molecular representation emerges as a key determinant in TFM performance, with molecular foundation model embeddings and 2D descriptor sets both providing substantial gains over classic molecular fingerprints on many tasks. These results suggest that in-context learning with TFMs provides a highly accurate and cost-efficient alternative for property prediction in practical applications.