# REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers

arXiv: https://arxiv.org/abs/2302.02041 · HTML: https://mlsanity.com/paper/2302.02041/

## Key facts

- Cited by 94 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 44 references that are in the map.
- First posted on arXiv 4 Feb 2023.

## Builds on (most cited first, 20 of 20)

- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [Tabular Data: Deep Learning is Not All You Need](https://mlsanity.com/paper/2106.03253/) (2021)
- [Are Transformers universal approximators of sequence-to-sequence functions?](https://mlsanity.com/paper/1912.10077/) (2019)
- [Synthetic Data for Text Localisation in Natural Images](https://mlsanity.com/paper/1604.06646/) (2016)
- [TabDDPM: Modelling Tabular Data with Diffusion Models](https://mlsanity.com/paper/2209.15421/) (2022)
- [Language Models are Realistic Tabular Data Generators](https://mlsanity.com/paper/2210.06280/) (2022)
- [Data Synthesis based on Generative Adversarial Networks](https://mlsanity.com/paper/1806.03384/) (2018)
- [CTAB-GAN: Effective Table Data Synthesizing](https://mlsanity.com/paper/2102.08369/) (2021)
- [Tabular Transformers for Modeling Multivariate Time Series](https://mlsanity.com/paper/2011.01843/) (2020)
- [Macro F1 and Macro F1](https://mlsanity.com/paper/1911.03347/) (2019)
- [CTAB-GAN+: Enhancing Tabular Data Synthesis](https://mlsanity.com/paper/2204.00401/) (2022)
- [Differential Privacy and Machine Learning: a Survey and Review](https://mlsanity.com/paper/1412.7584/) (2014)
- [A Non-Parametric Test to Detect Data-Copying in Generative Models](https://mlsanity.com/paper/2004.05675/) (2020)
- [Row Conditional-TGAN for generating synthetic relational databases](https://mlsanity.com/paper/2211.07588/) (2022)
- [Deep learning for comprehensive forecasting of Alzheimer's Disease progression](https://mlsanity.com/paper/1807.03876/) (2018)
- [Synthesising Multi-Modal Minority Samples for Tabular Data](https://mlsanity.com/paper/2105.08204/) (2021)
- [A review of homomorphic encryption and software tools for encrypted statistical machine learning](https://mlsanity.com/paper/1508.06574/) (2015)

## Built on by (most cited first, 30 of 94)

- [Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey](https://mlsanity.com/paper/2402.17944/) (2024)
- [TabuLa: Harnessing Language Models for Tabular Data Synthesis](https://mlsanity.com/paper/2310.12746/) (2023)
- [TabMT: Generating tabular data with masked transformers](https://mlsanity.com/paper/2312.06089/) (2023)
- [TabPFGen -- Tabular Data Generation with TabPFN](https://mlsanity.com/paper/2406.05216/) (2024)
- [A Comprehensive Survey of Synthetic Tabular Data Generation](https://mlsanity.com/paper/2504.16506/) (2025)
- [Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolution](https://mlsanity.com/paper/2408.10548/) (2024)
- [PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models](https://mlsanity.com/paper/2602.04029/) (2026)
- [Systematic Assessment of Tabular Data Synthesis](https://mlsanity.com/paper/2402.06806/) (2024)
- [TabSTAR: A Tabular Foundation Model for Tabular Data with Text Fields](https://mlsanity.com/paper/2505.18125/) (2025)
- [HARMONIC: Harnessing LLMs for Tabular Data Synthesis and Privacy Protection](https://mlsanity.com/paper/2408.02927/) (2024)
- [LLM/Agent-as-Data-Analyst: A Survey](https://mlsanity.com/paper/2509.23988/) (2025)
- [The Inadequacy of Similarity-based Privacy Metrics: Privacy Attacks against "Truly Anonymous" Synthetic Datasets](https://mlsanity.com/paper/2312.05114/) (2023)
- [TabularARGN: A Flexible and Efficient Auto-Regressive Framework for Generating High-Fidelity Synthetic Data](https://mlsanity.com/paper/2501.12012/) (2025)
- [One Transformer for All Time Series: Representing and Training with Time-Dependent Heterogeneous Tabular Data](https://mlsanity.com/paper/2302.06375/) (2023)
- [A Case Study Exploring the Current Landscape of Synthetic Medical Record Generation with Commercial LLMs](https://mlsanity.com/paper/2504.14657/) (2025)
- [Data Plagiarism Index: Characterizing the Privacy Risk of Data-Copying in Tabular Generative Models](https://mlsanity.com/paper/2406.13012/) (2024)
- [Why LLMs Are Bad at Synthetic Table Generation (and what to do about it)](https://mlsanity.com/paper/2406.14541/) (2024)
- [TabGen-ICL: Residual-Aware In-Context Example Selection for Tabular Data Generation](https://mlsanity.com/paper/2502.16414/) (2025)
- [LLM-TabLogic: Preserving Inter-Column Logical Relationships in Synthetic Tabular Data via Prompt-Guided Latent Diffusion](https://mlsanity.com/paper/2503.02161/) (2025)
- [Navigating Tabular Data Synthesis Research: Understanding User Needs and Tool Capabilities](https://mlsanity.com/paper/2405.20959/) (2024)
- [LaTable: Towards Large Tabular Models](https://mlsanity.com/paper/2406.17673/) (2024)
- [MALLM-GAN: Multi-Agent Large Language Model as Generative Adversarial Network for Synthesizing Tabular Data](https://mlsanity.com/paper/2406.10521/) (2024)
- [Benchmarking the Fidelity and Utility of Synthetic Relational Data](https://mlsanity.com/paper/2410.03411/) (2024)
- [A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data](https://mlsanity.com/paper/2601.17717/) (2026)
- [GeoFormer: Predicting Human Mobility using Generative Pre-trained Transformer (GPT)](https://mlsanity.com/paper/2311.05092/) (2023)
- [Improve Fidelity and Utility of Synthetic Credit Card Transaction Time Series from Data-centric Perspective](https://mlsanity.com/paper/2401.00965/) (2024)
- [Diffusion Transformers for Tabular Data Time Series Generation](https://mlsanity.com/paper/2504.07566/) (2025)
- [Privacy Auditing Synthetic Data Release through Local Likelihood Attacks](https://mlsanity.com/paper/2508.21146/) (2025)
- [Privacy Measurement in Tabular Synthetic Data: State of the Art and Future Research Directions](https://mlsanity.com/paper/2311.17453/) (2023)
- [Synthetic Data from Diffusion Models Improve Drug Discovery Prediction](https://mlsanity.com/paper/2405.03799/) (2024)

## Abstract (arXiv)

Tabular data is a common form of organizing data. Multiple models are available to generate synthetic tabular datasets where observations are independent, but few have the ability to produce relational datasets. Modeling relational data is challenging as it requires modeling both a "parent" table and its relationships across tables. We introduce REaLTabFormer (Realistic Relational and Tabular Transformer), a tabular and relational synthetic data generation model. It first creates a parent table using an autoregressive GPT-2 model, then generates the relational dataset conditioned on the parent table using a sequence-to-sequence (Seq2Seq) model. We implement target masking to prevent data copying and propose the Qδ statistic and statistical bootstrapping to detect overfitting. Experiments using real-world datasets show that REaLTabFormer captures the relational structure better than a baseline model. REaLTabFormer also achieves state-of-the-art results on prediction tasks, "out-of-the-box", for large non-relational datasets without needing fine-tuning.