# TabPFGen -- Tabular Data Generation with TabPFN

arXiv: https://arxiv.org/abs/2406.05216 · HTML: https://mlsanity.com/paper/2406.05216/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 42 references that are in the map.
- First posted on arXiv 7 Jun 2024.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Implicit Behavioral Cloning](https://mlsanity.com/paper/2109.00137/) (2021)
- [Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One](https://mlsanity.com/paper/1912.03263/) (2019)
- [TabDDPM: Modelling Tabular Data with Diffusion Models](https://mlsanity.com/paper/2209.15421/) (2022)
- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [Language Models are Realistic Tabular Data Generators](https://mlsanity.com/paper/2210.06280/) (2022)
- [A Theory of Generative ConvNet](https://mlsanity.com/paper/1602.03264/) (2016)
- [REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers](https://mlsanity.com/paper/2302.02041/) (2023)
- [Diffusion models for missing value imputation in tabular data](https://mlsanity.com/paper/2210.17128/) (2022)
- [Synthcity: facilitating innovative use cases of synthetic data in different data modalities](https://mlsanity.com/paper/2301.07573/) (2023)
- [Trust Issues: Uncertainty Estimation Does Not Enable Reliable OOD Detection On Medical Tabular Data](https://mlsanity.com/paper/2011.03274/) (2020)
- [Sequential Deep Learning for Credit Risk Monitoring with Tabular Financial Data](https://mlsanity.com/paper/2012.15330/) (2020)
- [Survey of resampling techniques for improving classification performance in unbalanced datasets](https://mlsanity.com/paper/1608.06048/) (2016)
- [Learning Energy-Based Models as Generative ConvNets via Multi-grid Modeling and Sampling](https://mlsanity.com/paper/1709.08868/) (2017)
- [On the Usefulness of Synthetic Tabular Data Generation](https://mlsanity.com/paper/2306.15636/) (2023)
- [Generative Trees: Adversarial and Copycat](https://mlsanity.com/paper/2201.11205/) (2022)
- [Use Classifier as Generator](https://mlsanity.com/paper/2209.09210/) (2022)

## Built on by (most cited first, 28 of 28)

- [TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models](https://mlsanity.com/paper/2511.08667/) (2025)
- [TabDPT: Scaling Tabular Foundation Models on Real Data](https://mlsanity.com/paper/2410.18164/) (2024)
- [A Closer Look at TabPFN v2: Understanding Its Strengths and Extending Its Capabilities](https://mlsanity.com/paper/2502.17361/) (2025)
- [TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problems](https://mlsanity.com/paper/2502.02527/) (2025)
- [Representation Learning for Tabular Data: A Comprehensive Survey](https://mlsanity.com/paper/2504.16109/) (2025)
- [TabPFN-3: Technical Report](https://mlsanity.com/paper/2605.13986/) (2026)
- [TabEBM: A Tabular Data Augmentation Method with Distinct Class-Specific Energy-Based Models](https://mlsanity.com/paper/2409.16118/) (2024)
- [A Survey on Deep Learning Approaches for Tabular Data Generation: Utility, Alignment, Fidelity, Privacy, Diversity, and Beyond](https://mlsanity.com/paper/2503.05954/) (2025)
- [Effortless, Simulation-Efficient Bayesian Inference using Tabular Foundation Models](https://mlsanity.com/paper/2504.17660/) (2025)
- [TACTIC for Navigating the Unknown: Tabular Anomaly deteCTion via In-Context inference](https://mlsanity.com/paper/2603.14171/) (2026)
- [Unmasking Trees for Tabular Data](https://mlsanity.com/paper/2407.05593/) (2024)
- [When Tables Leak: Attacking String Memorization in LLM-Based Tabular Data Generation](https://mlsanity.com/paper/2512.08875/) (2025)
- [Improving TabPFN's Synthetic Data Generation by Integrating Causal Structure](https://mlsanity.com/paper/2603.10254/) (2026)
- [metabeta -- A fast neural model for Bayesian mixed-effects regression](https://mlsanity.com/paper/2510.07473/) (2025)
- [TREB: a BERT attempt for imputing tabular data imputation](https://mlsanity.com/paper/2410.00022/) (2024)
- [NRGBoost: Energy-Based Generative Boosted Trees](https://mlsanity.com/paper/2410.03535/) (2024)
- [TabPFN for Zero-shot Parametric Engineering Design Generation](https://mlsanity.com/paper/2602.02735/) (2026)
- [Privately Fine-Tuned LLMs Preserve Temporal Dynamics in Tabular Data](https://mlsanity.com/paper/2602.02766/) (2026)
- [A Sobering Look at Tabular Data Generation via Probabilistic Circuits](https://mlsanity.com/paper/2603.23016/) (2026)
- [GEAR: Generative Expansion and Real Anchoring for Two-Stage Distillation of Tabular Foundation Models](https://mlsanity.com/paper/2608.18849/) (2026)
- [Guided Data Generation for Understanding Model Behavior](https://mlsanity.com/paper/2502.06658/) (2025)
- [RN-F: A Novel Approach for Mitigating Contaminated Data in Large Language Models](https://mlsanity.com/paper/2505.13249/) (2025)
- [Privacy-Preserving Tabular Synthetic Data Generation Using TabularARGN](https://mlsanity.com/paper/2508.06647/) (2025)
- [GEM-T: Generative Tabular Data via Fitting Moments](https://mlsanity.com/paper/2509.17752/) (2025)
- [Constrained Triple-Head Price Optimization (C3PO) Network for Discrete-Choice Models](https://mlsanity.com/paper/2605.06941/) (2026)
- [Correcting Class Imbalance in Prior-Data Fitted Networks for Tabular Classification](https://mlsanity.com/paper/2605.21742/) (2026)
- [CDMD: A Cross-Dataset Mixed-Type Diffusion Model for Tabular Data](https://mlsanity.com/paper/2609.39124/) (2026)
- [TabMosaic: Unified In-Context Diffusion for Tabular Generative Tasks](https://mlsanity.com/paper/2610.15699/) (2026)

## Abstract (arXiv)

Advances in deep generative modelling have not translated well to tabular data. We argue that this is caused by a mismatch in structure between popular generative models and discriminative models of tabular data. We thus devise a technique to turn TabPFN -- a highly performant transformer initially designed for in-context discriminative tabular tasks -- into an energy-based generative model, which we dub TabPFGen. This novel framework leverages the pre-trained TabPFN as part of the energy function and does not require any additional training or hyperparameter tuning, thus inheriting TabPFN's in-context learning capability. We can sample from TabPFGen analogously to other energy-based models. We demonstrate strong results on standard generative modelling tasks, including data augmentation, class-balancing, and imputation, unlocking a new frontier of tabular data generation.