# TabGen-ICL: Residual-Aware In-Context Example Selection for Tabular Data Generation

arXiv: https://arxiv.org/abs/2502.16414 · HTML: https://mlsanity.com/paper/2502.16414/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 36 references that are in the map.
- First posted on arXiv 23 Feb 2025.

## Builds on (most cited first, 19 of 19)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [TabDDPM: Modelling Tabular Data with Diffusion Models](https://mlsanity.com/paper/2209.15421/) (2022)
- [Language Models are Realistic Tabular Data Generators](https://mlsanity.com/paper/2210.06280/) (2022)
- [Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space](https://mlsanity.com/paper/2310.09656/) (2023)
- [REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers](https://mlsanity.com/paper/2302.02041/) (2023)
- [STaSy: Score-based Tabular data Synthesis](https://mlsanity.com/paper/2210.04018/) (2022)
- [CoDi: Co-evolving Contrastive Diffusion Models for Mixed-type Tabular Synthesis](https://mlsanity.com/paper/2304.12654/) (2023)
- [Diffusion models for missing value imputation in tabular data](https://mlsanity.com/paper/2210.17128/) (2022)
- [On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means](https://mlsanity.com/paper/1904.04017/) (2019)
- [Generative Table Pre-training Empowers Models for Tabular Prediction](https://mlsanity.com/paper/2305.09696/) (2023)
- [Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimes](https://mlsanity.com/paper/2312.12112/) (2023)
- [Tabular Representation, Noisy Operators, and Impacts on Table Structure Understanding Tasks in LLMs](https://mlsanity.com/paper/2310.10358/) (2023)
- ["We Need Structured Output": Towards User-centered Constraints on Large Language Model Output](https://mlsanity.com/paper/2404.07362/) (2024)
- [TabMT: Generating tabular data with masked transformers](https://mlsanity.com/paper/2312.06089/) (2023)
- [Google Dataset Search by the Numbers](https://mlsanity.com/paper/2006.06894/) (2020)

## Built on by (most cited first, 10 of 10)

- [Scaling Laws for Many-Shot In-Context Learning with Self-Generated Annotations](https://mlsanity.com/paper/2503.03062/) (2025)
- [A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data](https://mlsanity.com/paper/2601.17717/) (2026)
- [TAGAL: Tabular Data Generation using Agentic LLM Methods](https://mlsanity.com/paper/2509.04152/) (2025)
- [MUSE: Model-Agnostic Tabular Watermarking via Multi-Sample Selection](https://mlsanity.com/paper/2505.24267/) (2025)
- [SAGE: Sparse Adaptive Guidance for Dependency-Aware Tabular Data Generation](https://mlsanity.com/paper/2604.24368/) (2026)
- [StructSynth: Dependency Graphs as Generation Plans for Low-Data Tabular Synthesis with Language Models](https://mlsanity.com/paper/2508.02601/) (2025)
- [LLM-Driven Performance-Space Augmentation for Meta-Learning-Based Algorithm Selection](https://mlsanity.com/paper/2605.09518/) (2026)
- [Algorithmic Recourse of In-Context Learning for Tabular Data](https://mlsanity.com/paper/2605.31272/) (2026)
- [Cross-Domain Feature Expansion for Tabular Medical Data via Knowledge Graphs Injection](https://mlsanity.com/paper/2606.31171/) (2026)
- [LAB-Tab: LLM-Augmented Bayesian Network Adaptation for Few-Shot Tabular Generation](https://mlsanity.com/paper/2608.01879/) (2026)

## Abstract (arXiv)

Large Language models (LLMs) have achieved encouraging results in tabular data generation. However, existing approaches require fine-tuning, which is computationally expensive. This paper explores an alternative: prompting a fixed LLM with in-context examples. We observe that using randomly selected in-context examples hampers the LLM's performance, resulting in sub-optimal generation quality. To address this, we propose a novel in-context learning framework: TabGen-ICL, to enhance the in-context learning ability of LLMs for tabular data generation. TabGen-ICL operates iteratively, retrieving a subset of real samples that represent the residual between currently generated samples and true data distributions. This approach serves two purposes: locally, it provides more effective in-context learning examples for the LLM in each iteration; globally, it progressively narrows the gap between generated and real data. Extensive experiments on five real-world tabular datasets demonstrate that TabGen-ICL significantly outperforms the random selection strategy. Specifically, it reduces the error rate by a margin of 3.5%-42.2% on fidelity metrics. We demonstrate for the first time that prompting a fixed LLM can yield high-quality synthetic tabular data. The code is provided in the hrefhttps://github.com/fangliancheng/TabGEN-ICLlink.