# Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models

arXiv: https://arxiv.org/abs/2510.21204 · HTML: https://mlsanity.com/paper/2510.21204/

## Key facts

- Cited by 77 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 40 references that are in the map.
- First posted on arXiv 24 Oct 2025.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data](https://mlsanity.com/paper/2003.06505/) (2020)
- [SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training](https://mlsanity.com/paper/2106.01342/) (2021)
- [TabICL: A Tabular Foundation Model for In-Context Learning on Large Data](https://mlsanity.com/paper/2502.05564/) (2025)
- [TabLLM: Few-shot Classification of Tabular Data with Large Language Models](https://mlsanity.com/paper/2210.10723/) (2022)
- [When Do Neural Nets Outperform Boosted Trees on Tabular Data?](https://mlsanity.com/paper/2305.02997/) (2023)
- [TabArena: A Living Benchmark for Machine Learning on Tabular Data](https://mlsanity.com/paper/2506.16791/) (2025)
- [Well-tuned Simple Nets Excel on Tabular Datasets](https://mlsanity.com/paper/2106.11189/) (2021)
- [TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensembling](https://mlsanity.com/paper/2410.24210/) (2024)
- [Why Tabular Foundation Models Should Be a Research Priority](https://mlsanity.com/paper/2405.01147/) (2024)
- [Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Data](https://mlsanity.com/paper/2407.04491/) (2024)
- [XTab: Cross-table Pretraining for Tabular Transformers](https://mlsanity.com/paper/2305.06090/) (2023)
- [CARTE: Pretraining and Transfer for Tabular Learning](https://mlsanity.com/paper/2402.16785/) (2024)
- [Large Scale Transfer Learning for Tabular Data via Language Modeling](https://mlsanity.com/paper/2406.12031/) (2024)
- [Making Pre-trained Language Models Great on Tabular Prediction](https://mlsanity.com/paper/2403.01841/) (2024)
- [AMLB: an AutoML Benchmark](https://mlsanity.com/paper/2207.12560/) (2022)
- [TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks](https://mlsanity.com/paper/2402.11137/) (2024)
- [TabRepo: A Large Scale Repository of Tabular Model Evaluations and its AutoML Applications](https://mlsanity.com/paper/2311.02971/) (2023)
- [Mixture of In-Context Prompters for Tabular PFNs](https://mlsanity.com/paper/2405.16156/) (2024)
- [UniTabE: A Universal Pretraining Protocol for Tabular Foundation Model in Data Science](https://mlsanity.com/paper/2307.09249/) (2023)
- [From Supervised to Generative: A Novel Paradigm for Tabular Deep Learning with Large Language Models](https://mlsanity.com/paper/2310.07338/) (2023)
- [Fine-tuned In-Context Learning Transformers are Excellent Tabular Data Classifiers](https://mlsanity.com/paper/2405.13396/) (2024)
- [HyperFast: Instant Classification for Tabular Data](https://mlsanity.com/paper/2402.14335/) (2024)
- [Towards Cross-Table Masked Pretraining for Web Data Mining](https://mlsanity.com/paper/2307.04308/) (2023)
- [In-Context Data Distillation with TabPFN](https://mlsanity.com/paper/2402.06971/) (2024)

## Built on by (most cited first, 30 of 77)

- [TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models](https://mlsanity.com/paper/2511.08667/) (2025)
- [PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models](https://mlsanity.com/paper/2602.04029/) (2026)
- [KumoRFM-2: Scaling Foundation Models for Relational Learning](https://mlsanity.com/paper/2604.12596/) (2026)
- [Relational In-Context Learning via Synthetic Pre-training with Structural Prior](https://mlsanity.com/paper/2603.03805/) (2026)
- [No Need to Train Your RDB Foundation Model](https://mlsanity.com/paper/2602.13697/) (2026)
- [Exploring Fine-Tuning for Tabular Foundation Models](https://mlsanity.com/paper/2601.09654/) (2026)
- [Tabular Foundation Models Can Do Survival Analysis](https://mlsanity.com/paper/2601.22259/) (2026)
- [From Zero to Hero: Advancing Zero-Shot Foundation Models for Tabular Outlier Detection](https://mlsanity.com/paper/2602.03018/) (2026)
- [LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models](https://mlsanity.com/paper/2606.04485/) (2026)
- [LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence](https://mlsanity.com/paper/2609.17488/) (2026)
- [Use What You Know: Causal Foundation Models with Partial Graphs](https://mlsanity.com/paper/2602.14972/) (2026)
- [TACTIC for Navigating the Unknown: Tabular Anomaly deteCTion via In-Context inference](https://mlsanity.com/paper/2603.14171/) (2026)
- [Auditing and Fixing Economic Validity in Tabular Foundation Models for Discrete Choice](https://mlsanity.com/paper/2605.26559/) (2026)
- [TabDPT-Turbo: Efficient In-Context Learning for Tabular Prediction](https://mlsanity.com/paper/2608.01400/) (2026)
- [TabClustPFN: A Prior-data Fitted Network for Tabular Data Clustering](https://mlsanity.com/paper/2601.21656/) (2026)
- [MacrOData: New Benchmarks of Thousands of Datasets for Tabular Outlier Detection](https://mlsanity.com/paper/2602.09329/) (2026)
- [Interpretable Tabular Foundation Models via In-Context Kernel Regression](https://mlsanity.com/paper/2602.02162/) (2026)
- [Comparing Task-Agnostic Embedding Models for Tabular Data](https://mlsanity.com/paper/2511.14276/) (2025)
- [Distributional Regression with Tabular Foundation Models: Evaluating Probabilistic Predictions via Proper Scoring Rules](https://mlsanity.com/paper/2603.08206/) (2026)
- [RamanBench: A Large-Scale Benchmark for Machine Learning on Raman Spectroscopy](https://mlsanity.com/paper/2605.02003/) (2026)
- [VIP-COP: Context Optimization for Tabular Foundation Models](https://mlsanity.com/paper/2605.12904/) (2026)
- [SurvivalPFN: Amortizing Survival Prediction via In-Context Bayesian Inference](https://mlsanity.com/paper/2605.15488/) (2026)
- [Embedding Foundation Model Predictions in Discrete-Choice Models with Structural Guarantees](https://mlsanity.com/paper/2606.26432/) (2026)
- [FIRE: Multi-fidelity Regression with Distribution-conditioned In-context Learning using Tabular Foundation Models](https://mlsanity.com/paper/2601.22371/) (2026)
- [MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image](https://mlsanity.com/paper/2605.10616/) (2026)
- [CalArena: A Large-Scale Post-Hoc Calibration Benchmark](https://mlsanity.com/paper/2605.30188/) (2026)
- [TabPrep: Closing the Feature Engineering Gap in Tabular Benchmarks](https://mlsanity.com/paper/2606.02384/) (2026)
- [CRUMB: Efficient Prior Fitted Network Inference via Distributionally Matched Context Batching](https://mlsanity.com/paper/2606.11473/) (2026)
- [Understanding the Surprising Generalization Properties of Tabular Foundation Models](https://mlsanity.com/paper/2608.17957/) (2026)
- [iLTM: Integrated Large Tabular Model](https://mlsanity.com/paper/2511.15941/) (2025)

## Abstract (arXiv)

Since the seminal work of TabPFN, research on tabular foundation models (TFMs) based on in-context learning (ICL) has challenged long-standing paradigms in machine learning. Without seeing any real-world data, models pretrained on purely synthetic datasets generalize remarkably well across diverse datasets, often using only a moderate number of in-context examples. This shifts the focus in tabular machine learning from model architecture design to the design of synthetic datasets, or, more precisely, to the prior distributions that generate them. Yet the guiding principles for prior design remain poorly understood. This work marks the first attempt to address the gap. We systematically investigate and identify key properties of synthetic priors that allow pretrained TFMs to generalize well. Based on these insights, we introduce Mitra, a TFM trained on a curated mixture of synthetic priors selected for their diversity, distinctiveness, and performance on real-world tabular data. Mitra consistently outperforms state-of-the-art TFMs, such as TabPFNv2 and TabICL, across both classification and regression benchmarks, with better sample efficiency.