# Generative Modeling of Complex Data

arXiv: https://arxiv.org/abs/2202.02145 · HTML: https://mlsanity.com/paper/2202.02145/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 4 Feb 2022.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [Data Synthesis based on Generative Adversarial Networks](https://mlsanity.com/paper/1806.03384/) (2018)
- [CTAB-GAN: Effective Table Data Synthesizing](https://mlsanity.com/paper/2102.08369/) (2021)
- [Tabular Transformers for Modeling Multivariate Time Series](https://mlsanity.com/paper/2011.01843/) (2020)
- [VAEM: a Deep Generative Model for Heterogeneous Mixed Type Data](https://mlsanity.com/paper/2006.11941/) (2020)
- [Copula Flows for Synthetic Data Generation](https://mlsanity.com/paper/2101.00598/) (2021)
- [Generating Multi-Categorical Samples with Generative Adversarial Networks](https://mlsanity.com/paper/1807.01202/) (2018)
- [SYNC: A Copula based Framework for Generating Synthetic Data from Aggregated Sources](https://mlsanity.com/paper/2009.09471/) (2020)

## Built on by (most cited first, 3 of 3)

- [DP-TBART: A Transformer-based Autoregressive Model for Differentially Private Tabular Data Generation](https://mlsanity.com/paper/2307.10430/) (2023)
- [Privately generating tabular data using language models](https://mlsanity.com/paper/2306.04803/) (2023)
- [Qrlew: Rewriting SQL into Differentially Private SQL](https://mlsanity.com/paper/2401.06273/) (2024)

## Abstract (arXiv)

In recent years, several models have improved the capacity to generate synthetic tabular datasets. However, such models focus on synthesizing simple columnar tables and are not useable on real-life data with complex structures. This paper puts forward a generic framework to synthesize more complex data structures with composite and nested types. It then proposes one practical implementation, built with causal transformers, for struct (mappings of types) and lists (repeated instances of a type). The results on standard benchmark datasets show that such implementation consistently outperforms current state-of-the-art models both in terms of machine learning utility and statistical similarity. Moreover, it shows very strong results on two complex hierarchical datasets with multiple nesting and sparse data, that were previously out of reach.