# One Pipeline, Many Transformers: Pattern-Specific Imputation Specialists for Tabular Missing Data

arXiv: https://arxiv.org/abs/2510.02625 · HTML: https://mlsanity.com/paper/2510.02625/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 66 references that are in the map.
- First posted on arXiv 3 Oct 2025.

## Builds on (most cited first, 23 of 23)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [GAIN: Missing Data Imputation using Generative Adversarial Nets](https://mlsanity.com/paper/1806.02920/) (2018)
- [TabICL: A Tabular Foundation Model for In-Context Learning on Large Data](https://mlsanity.com/paper/2502.05564/) (2025)
- [Recommendations as Treatments: Debiasing Learning and Evaluation](https://mlsanity.com/paper/1602.05352/) (2016)
- [TabDPT: Scaling Tabular Foundation Models on Real Data](https://mlsanity.com/paper/2410.18164/) (2024)
- [Matrix Completion and Low-Rank SVD via Fast Alternating Least Squares](https://mlsanity.com/paper/1410.2596/) (2014)
- [HyperImpute: Generalized Iterative Imputation with Automatic Model Selection](https://mlsanity.com/paper/2206.07769/) (2022)
- [not-MIWAE: Deep Generative Modelling with Missing not at Random Data](https://mlsanity.com/paper/2006.12871/) (2020)
- [Generating and Imputing Tabular Data via Diffusion and Flow-based Gradient-Boosted Trees](https://mlsanity.com/paper/2309.09968/) (2023)
- [Causal Matrix Completion](https://mlsanity.com/paper/2109.15154/) (2021)
- [ReMasker: Imputing Tabular Data with Masked Autoencoding](https://mlsanity.com/paper/2309.13793/) (2023)
- [Learning from data with structured missingness](https://mlsanity.com/paper/2304.01429/) (2023)
- [Imputation and low-rank estimation with Missing Not At Random data](https://mlsanity.com/paper/1812.11409/) (2018)
- [DiffPuter: Empowering Diffusion Models for Missing Data Imputation](https://mlsanity.com/paper/2405.20690/) (2024)
- [TabFlex: Scaling Tabular Learning to Millions with Linear Attention](https://mlsanity.com/paper/2506.05584/) (2025)
- [CACTI: Leveraging Copy Masking and Contextual Information to Improve Tabular Data Imputation](https://mlsanity.com/paper/2506.02306/) (2025)
- [A Complete Characterisation of Structured Missingness](https://mlsanity.com/paper/2307.02650/) (2023)

## Built on by (most cited first, 4 of 4)

- [TabPFN-3: Technical Report](https://mlsanity.com/paper/2605.13986/) (2026)
- [Submodular Benchmark Selection](https://mlsanity.com/paper/2605.02209/) (2026)
- [Group-Aware Matrix Estimation and Latent Subspace Recovery](https://mlsanity.com/paper/2605.20559/) (2026)
- [MatrixFormer: A Foundation Model for Matrix Completion](https://mlsanity.com/paper/2610.06751/) (2026)

## Abstract (arXiv)

Missing data in tabular datasets forces practitioners into a hard choice: deploy a general-purpose imputer that may perform poorly for the problem at hand, or wait for someone to design a specialized algorithm. This problem is worsened by the fact that real-world missingness rarely satisfies the textbook missing completely at random (MCAR) assumption, as entries are often missing not at random (MNAR), where the probability of being observed depends on the missing data itself. We collapse this trade-off into a single pre-training pipeline that builds transformer-based imputation specialists through three components: an entry-wise featurization that recasts imputation as supervised prediction over row--column context, a synthetic data generator with pluggable missingness modules, and prior-data fitting on millions of synthetic tables. Swapping only the missingness module, with no changes to architecture, loss, or training, yields a pattern-specific specialist that outperforms methods purpose-built for that pattern. We validate this on MissBench, a new benchmark of 42 OpenML datasets and 11 missingness patterns (including 9 MNAR variants) spanning medicine, finance, and engineering. Remarkably, training exclusively on MCAR yields a default model---TabImpute---robust across all tested patterns. In addition, the pattern-specific specialists produced by our pipeline outperform the 14 established baselines on their target patterns. We open-source the pipeline, models, and benchmark.