mlsanityData to 14 Oct 2026

TabMT: Generating tabular data with masked transformers

arXiv 2312.06089 ↗First posted 11 Dec 2023cs.LG cs.AIcited by 36

Not on the map. Why?

Abstract (arXiv)

Autoregressive and Masked Transformers are incredibly effective as generative models and classifiers. While these models are most prevalent in NLP, they also exhibit strong performance in other domains, such as vision. This work contributes to the exploration of transformer-based models in synthetic data generation for diverse application domains. In this paper, we present TabMT, a novel Masked Transformer design for generating synthetic tabular data. TabMT effectively addresses the unique challenges posed by heterogeneous data fields and is natively able to handle missing data. Our design leverages improved masking techniques to allow for generation and demonstrates state-of-the-art performance from extremely small to extremely large tabular datasets. We evaluate TabMT for privacy-focused applications and find that it is able to generate high quality data with superior privacy tradeoffs.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Builds on 24 of its 37 references that are in the map. First posted on arXiv 11 Dec 2023.

Builds on

24 of 37 references.

References · 24

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  5. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  6. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  7. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  8. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  9. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  10. 2023Extracting Training Data from Diffusion Modelscited by 732
  11. 2019Modeling Tabular data using Conditional GANcited by 699
  12. 2022TabDDPM: Modelling Tabular Data with Diffusion Modelscited by 377
12 more
  1. 2022Language Models are Realistic Tabular Data Generatorscited by 294
  2. 2022Masked Autoencoders that Listencited by 273
  3. 2018CatBoost: gradient boosting with categorical features supportcited by 208
  4. 2020Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualizationcited by 173
  5. 2021TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Upcited by 162
  6. 2021CTAB-GAN: Effective Table Data Synthesizingcited by 156
  7. 2023REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformerscited by 94
  8. 2022STaSy: Score-based Tabular data Synthesiscited by 86
  9. 2020Tabular Transformers for Modeling Multivariate Time Seriescited by 77
  10. 2022CTAB-GAN+: Enhancing Tabular Data Synthesiscited by 70
  11. 2020Conditional Wasserstein GAN-based Oversampling of Tabular Data for Imbalanced Learningcited by 67
  12. 2018Flow-based Network Traffic Generation using Generative Adversarial Networkscited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 15; 2025: 17; 2026: 4.

36 papers.

Sort:

By area:

  1. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 159
  2. 2025A Comprehensive Survey of Synthetic Tabular Data Generationcited by 28
  3. 2025Large Language Models are Good Relational Learnerscited by 24
  4. 2024Differentially Private Tabular Data Synthesis using Large Language Modelscited by 16
  5. 2025TabularARGN: A Flexible and Efficient Auto-Regressive Framework for Generating High-Fidelity Synthetic Datacited by 14
  6. 2025TabGen-ICL: Residual-Aware In-Context Example Selection for Tabular Data Generationcited by 10
  7. 2024Attention versus Contrastive Learning of Tabular Data -- A Data-centric Benchmarkingcited by 9
  8. 2024Generating Realistic Tabular Data with Large Language Modelscited by 9
  9. 2024Causality for Tabular Data Synthesis: A High-Order Structure Causal Benchmark Frameworkcited by 8
  10. 2024Synthetic Tabular Data Generation for Imbalanced Classification: The Surprising Effectiveness of an Overlap Classcited by 8
  11. 2025Not All Features Deserve Attention: Graph-Guided Dependency Learning for Tabular Data Generation with Language Modelscited by 8
  12. 2024MALLM-GAN: Multi-Agent Large Language Model as Generative Adversarial Network for Synthesizing Tabular Datacited by 7
24 more
  1. 2024Deep Feature Embedding for Tabular Datacited by 7
  2. 2024A Context-Aware Approach for Enhancing Data Imputation with Pre-trained Language Modelscited by 5
  3. 2025Diffusion Transformers for Tabular Data Time Series Generationcited by 5
  4. 2025Table Question Answering in the Era of Large Language Models: A Comprehensive Survey of Tasks, Methods, and Evaluationcited by 5
  5. 2024Unmasking Trees for Tabular Datacited by 4
  6. 2024Diffusion-nested Auto-Regressive Synthesis of Heterogeneous Tabular Datacited by 4
  7. 2024iTBLS: A Dataset of Interactive Conversations Over Tabular Informationcited by 3
  8. 2025MaskSDM with Shapley values to improve flexibility, robustness, and explainability in species distribution modelingcited by 3
  9. 2025Graph-Conditional Flow Matching for Relational Data Generationcited by 3
  10. 2025TabTreeFormer: Tabular Data Generation Using Hybrid Tree-Transformercited by 2
  11. 2025A Generalized Theory of Mixup for Structure-Preserving Synthetic Datacited by 2
  12. 2024Masked Language Modeling Becomes Conditional Density Estimation for Tabular Data Synthesiscited by 1
  13. 2024TAEGAN: Generating Synthetic Tabular Data For Data Augmentationcited by 1
  14. 2024NRGBoost: Energy-Based Generative Boosted Treescited by 1
  15. 2025MUSE: Model-Agnostic Tabular Watermarking via Multi-Sample Selectioncited by 1
  16. 2025Democratizing Tabular Data Access with an Openunicodex2013Source Syntheticunicodex2013Data SDKcited by 1
  17. 2025Towards Universal Debiasing for Language Models-based Tabular Data Generationcited by 1
  18. 2025Robust Spectral Watermark for Synthetic Tabular Datacited by 1
  19. 2025Taming Data Challenges in ML-based Security Tasks Using Generative AIcited by 0
  20. 2025Privacy-Preserving Tabular Synthetic Data Generation Using TabularARGNcited by 0
  21. 2026Exploring Transformer Placement in Variational Autoencoders for Tabular Data Generationcited by 0
  22. 2026Quantifying Membership Disclosure Risk for Tabular Synthetic Data Using Kernel Density Estimatorscited by 0
  23. 2026PATE-TabTransGAN: Differentially Private Synthetic Tabular Data Generation via Transformer-Based Student Discriminationcited by 0
  24. 2026MIND: Marginal-Invariant Neural Dependency Diffusion for Mixed-Type Tabular Generationcited by 0