mlsanityData to 14 Oct 2026

Do Tabular Foundation Models Know Physics? Contamination, Units, and the Deterministic Limit

arXiv 2609.02766 ↗First posted 2 Sep 2026cs.LG astro-ph.IMcited by 0

Not on the map. Why?

Abstract (arXiv)

Tabular foundation models (TFMs) learn to fill in tables the way language models fill in text, and tables are arguably the format in which most physical measurement arrives. Did they learn any physics in the process? They are Bayesian by construction, so the question is what their prior contains. We probe it directly, evaluating four of them (TabPFN-3, TabICLv2, TabDPT and Real-TabPFN-2.5) against six baselines on datasets sampled from 316 physical equations, in and out of domain. TFMs dominate, out of the box and after tuning. But we show that their prior can represent neither a noiseless mechanism nor physical units, which is why they interpolate physics without yet being able to act as physical models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 27 references that are in the map. First posted on arXiv 2 Sep 2026.

Builds on

15 of 27 references.

References · 15

  1. 2021Revisiting Deep Learning Models for Tabular Datacited by 740
  2. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  3. 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 548
  4. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 289
  5. 2021Contemporary Symbolic Regression Methods and their Relative Performancecited by 215
  6. 2025TabArena: A Living Benchmark for Machine Learning on Tabular Datacited by 176
  7. 2017PMLB: A Large Benchmark Suite for Machine Learning Evaluation and Comparisoncited by 140
  8. 2025TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Modelscited by 132
  9. 2024TabDPT: Scaling Tabular Foundation Models on Real Datacited by 113
  10. 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensemblingcited by 113
  11. 2024Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Datacited by 92
  12. 2025LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Modelscited by 71
3 more
  1. 2025Real-TabPFN: Improving Tabular Foundation Models via Continued Pre-training With Real-World Datacited by 41
  2. 2026Beyond IID: How General Are Tabular Foundation Models, Really?cited by 20
  3. 2025What exactly has TabPFN learned to do?cited by 5

Built on by

Not cited yet.