# TabuLa: Harnessing Language Models for Tabular Data Synthesis

arXiv: https://arxiv.org/abs/2310.12746 · HTML: https://mlsanity.com/paper/2310.12746/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Placed in Privacy-preserving tabular data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 16 of its 23 references that are in the map.
- First posted on arXiv 19 Oct 2023.

## Placed in

- Synthetic data and self-improvement for LLMs / [Privacy-preserving tabular data synthesis](https://mlsanity.com/synth/privacy-preserving-tabular-data-synthesis/)

## Builds on (most cited first, 16 of 16)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [TabDDPM: Modelling Tabular Data with Diffusion Models](https://mlsanity.com/paper/2209.15421/) (2022)
- [Language Models are Realistic Tabular Data Generators](https://mlsanity.com/paper/2210.06280/) (2022)
- [Data Synthesis based on Generative Adversarial Networks](https://mlsanity.com/paper/1806.03384/) (2018)
- [CTAB-GAN: Effective Table Data Synthesizing](https://mlsanity.com/paper/2102.08369/) (2021)
- [REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers](https://mlsanity.com/paper/2302.02041/) (2023)
- [Tabular Transformers for Modeling Multivariate Time Series](https://mlsanity.com/paper/2011.01843/) (2020)
- [CTAB-GAN+: Enhancing Tabular Data Synthesis](https://mlsanity.com/paper/2204.00401/) (2022)
- [SOS: Score-based Oversampling for Tabular Data](https://mlsanity.com/paper/2206.08555/) (2022)
- [Invertible Tabular GANs: Killing Two Birds with OneStone for Tabular Data Synthesis](https://mlsanity.com/paper/2202.03636/) (2022)
- [Airline Passenger Name Record Generation using Generative Adversarial Networks](https://mlsanity.com/paper/1807.06657/) (2018)
- [Generating Synthetic but Plausible Healthcare Record Datasets](https://mlsanity.com/paper/1807.01514/) (2018)
- [Permutation-Invariant Tabular Data Synthesis](https://mlsanity.com/paper/2211.09286/) (2022)
- [FCT-GAN: Enhancing Table Synthesis via Fourier Transform](https://mlsanity.com/paper/2210.06239/) (2022)

## Built on by (most cited first, 30 of 42)

- [On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://mlsanity.com/paper/2406.15126/) (2024)
- [Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey](https://mlsanity.com/paper/2402.17944/) (2024)
- [Why Tabular Foundation Models Should Be a Research Priority](https://mlsanity.com/paper/2405.01147/) (2024)
- [A Comprehensive Survey of Synthetic Tabular Data Generation](https://mlsanity.com/paper/2504.16506/) (2025)
- [Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolution](https://mlsanity.com/paper/2408.10548/) (2024)
- [HARMONIC: Harnessing LLMs for Tabular Data Synthesis and Privacy Protection](https://mlsanity.com/paper/2408.02927/) (2024)
- [CTSyn: A Foundation Model for Cross Tabular Data Generation](https://mlsanity.com/paper/2406.04619/) (2024)
- [A Case Study Exploring the Current Landscape of Synthetic Medical Record Generation with Commercial LLMs](https://mlsanity.com/paper/2504.14657/) (2025)
- [Why LLMs Are Bad at Synthetic Table Generation (and what to do about it)](https://mlsanity.com/paper/2406.14541/) (2024)
- [LLM-TabLogic: Preserving Inter-Column Logical Relationships in Synthetic Tabular Data via Prompt-Guided Latent Diffusion](https://mlsanity.com/paper/2503.02161/) (2025)
- [Navigating Tabular Data Synthesis Research: Understanding User Needs and Tool Capabilities](https://mlsanity.com/paper/2405.20959/) (2024)
- [LaTable: Towards Large Tabular Models](https://mlsanity.com/paper/2406.17673/) (2024)
- [MALLM-GAN: Multi-Agent Large Language Model as Generative Adversarial Network for Synthesizing Tabular Data](https://mlsanity.com/paper/2406.10521/) (2024)
- [Preserving logical and functional dependencies in synthetic tabular data](https://mlsanity.com/paper/2409.17684/) (2024)
- [A text-to-tabular approach to generate synthetic patient data using LLMs](https://mlsanity.com/paper/2412.05153/) (2024)
- [Tabby: A Language Model Architecture for Tabular and Structured Data Synthesis](https://mlsanity.com/paper/2503.02152/) (2025)
- [Generating Synthetic Data with Formal Privacy Guarantees: State of the Art and the Road Ahead](https://mlsanity.com/paper/2503.20846/) (2025)
- [TAGAL: Tabular Data Generation using Agentic LLM Methods](https://mlsanity.com/paper/2509.04152/) (2025)
- [Convex space learning for tabular synthetic data generation](https://mlsanity.com/paper/2407.09789/) (2024)
- [TIMeSynC: Temporal Intent Modelling with Synchronized Context Encodings for Financial Service Applications](https://mlsanity.com/paper/2410.12825/) (2024)
- [TabTreeFormer: Tabular Data Generation Using Hybrid Tree-Transformer](https://mlsanity.com/paper/2501.01216/) (2025)
- [Architectural and Inferential Inductive Biases For Exchangeable Sequence Modeling](https://mlsanity.com/paper/2503.01215/) (2025)
- [Dependency-aware synthetic tabular data generation](https://mlsanity.com/paper/2507.19211/) (2025)
- [Assessing Generative Models for Structured Data](https://mlsanity.com/paper/2503.20903/) (2025)
- [What's Wrong with Your Synthetic Tabular Data? Using Explainable AI to Evaluate Generative Models](https://mlsanity.com/paper/2504.20687/) (2025)
- [Towards Universal Debiasing for Language Models-based Tabular Data Generation](https://mlsanity.com/paper/2509.16475/) (2025)
- [PersonaLedger: Generating Realistic Financial Transactions with Persona Conditioned LLMs and Rule Grounded Feedback](https://mlsanity.com/paper/2601.03149/) (2026)
- [SAGE: Sparse Adaptive Guidance for Dependency-Aware Tabular Data Generation](https://mlsanity.com/paper/2604.24368/) (2026)
- [Categorical Prior Lock-in: Why In-Context Learning Fails for Structured Data](https://mlsanity.com/paper/2606.11961/) (2026)
- [A Note on Statistically Accurate Tabular Data Generation Using Large Language Models](https://mlsanity.com/paper/2505.02659/) (2025)

## Abstract (arXiv)

Tabular data synthesis is crucial for addressing privacy and security concerns in industries reliant on tabular data. While recent advancements adopt large language models (LLMs) for realistic tabular data generation, their long training times and limited reusability hinder practical applications. In this paper, we propose Tabula, a tabular data synthesizer that leverages the structure of LLM. Unlike state-of-the-art (SOTA) LLM-based tabular data synthesizers that rely on pre-trained LLMs, Tabula discards the pre-trained weights originally designed for natural language tasks, focusing instead on a tailored approach for tabular data. In addition, Tabula introduces a token sequence compression strategy that significantly reduces training time while maintaining data quality, alongside a novel token padding method that improves sequence alignment across training batches. Experiments on six datasets show that Tabula achieves superior synthetic data utility compared to current SOTA methods. Additionally, the results demonstrate that Tabula model trained on tabular datasets serves effectively as a foundational model for synthesizing new tabular datasets. Furthermore, the proposed padding method outperforms the conventional left and right padding strategies. Finally, the results highlight that Tabula averagely reduces training time per epoch by 46.2% compared to state-of-the-art LLM approaches while achieving higher data utility. Our code is available at https://github.com/zhao-zilong/Tabula