# Can Tabular Foundation Models Amortize Statistical Inference?

arXiv: https://arxiv.org/abs/2609.33114 · HTML: https://mlsanity.com/paper/2609.33114/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 27 Sep 2026.

## Builds on (most cited first, 21 of 21)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [TabICL: A Tabular Foundation Model for In-Context Learning on Large Data](https://mlsanity.com/paper/2502.05564/) (2025)
- [Distribution-Free, Risk-Controlling Prediction Sets](https://mlsanity.com/paper/2101.02703/) (2021)
- [OpenML Benchmarking Suites](https://mlsanity.com/paper/1708.03731/) (2017)
- [Conformal Prediction With Conditional Guarantees](https://mlsanity.com/paper/2305.12616/) (2023)
- [TabICLv2: A better, faster, scalable, and open tabular foundation model](https://mlsanity.com/paper/2602.11139/) (2026)
- [TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models](https://mlsanity.com/paper/2511.08667/) (2025)
- [LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence](https://mlsanity.com/paper/2509.03505/) (2025)
- [TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks](https://mlsanity.com/paper/2402.11137/) (2024)
- [Foundation Models for Causal Inference via Prior-Data Fitted Networks](https://mlsanity.com/paper/2506.10914/) (2025)
- [TabMGP: Martingale Posterior with TabPFN](https://mlsanity.com/paper/2510.25154/) (2025)
- [Uncertainty Decomposition for Bayes-Filtered Transformers via Bayesian Predictive Inference](https://mlsanity.com/paper/2602.04596/) (2026)
- [Uncertainty Quantification for Prior-Data Fitted Networks using Martingale Posteriors](https://mlsanity.com/paper/2505.11325/) (2025)
- [TabClustPFN: A Prior-data Fitted Network for Tabular Data Clustering](https://mlsanity.com/paper/2601.21656/) (2026)
- [Inference for Deep Neural Network Estimators in Generalized Nonparametric Models](https://mlsanity.com/paper/2504.09347/) (2025)
- [PFN-TS: Thompson Sampling for Contextual Bandits via Prior-Data Fitted Networks](https://mlsanity.com/paper/2605.10137/) (2026)
- [U-learning for Prediction Inference via Combinatory Multi-Subsampling: With Applications to LASSO and Neural Networks](https://mlsanity.com/paper/2407.15301/) (2024)
- [SOMTab: Set-Order Mamba for Efficient Tabular In-Context Learning](https://mlsanity.com/paper/2608.27882/) (2026)
- [Learning to Fluctuate: Statistical Foundations for Causal Tabular Pretraining](https://mlsanity.com/paper/2609.26290/) (2026)

## Abstract (arXiv)

For decades, statistical inference has largely been developed one problem at a time. Given a scientific target, such as a treatment effect or a regression function, statisticians design a problem-specific estimator together with a procedure for quantifying its uncertainty. This paper proposes a different paradigm. We focus on a classical problem in statistical inference, confidence interval construction, and develop TabCon, an amortized inference system built on a tabular foundation model that produces confidence intervals for new datasets through a simple forward pass. The key methodological ingredients of TabCon are a sparse mixture-of-experts architecture and reinforcement-learning-based post-training that calibrate the resulting confidence intervals to a desired coverage level. Across a wide range of benchmark datasets, TabCon attains near-nominal coverage while producing short confidence intervals. At inference time, it also offers considerably greater computational efficiency, running 50 times faster than the classical bootstrap procedure, even when the latter uses only 50 bootstrap samples.