# A Comprehensive Evaluation Framework for Synthetic Trip Data Generation in Public Transport

arXiv: https://arxiv.org/abs/2510.24375 · HTML: https://mlsanity.com/paper/2510.24375/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 56 references that are in the map.
- First posted on arXiv 28 Oct 2025.

## Builds on (most cited first, 25 of 25)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [Language Models are Realistic Tabular Data Generators](https://mlsanity.com/paper/2210.06280/) (2022)
- [Are GANs Created Equal? A Large-Scale Study](https://mlsanity.com/paper/1711.10337/) (2017)
- [TabuLa: Harnessing Language Models for Tabular Data Synthesis](https://mlsanity.com/paper/2310.12746/) (2023)
- [A Deep Generative Model for Feasible and Diverse Population Synthesis](https://mlsanity.com/paper/2208.01403/) (2022)
- [Diffusion Models for Intelligent Transportation Systems: A Survey](https://mlsanity.com/paper/2409.15816/) (2024)
- [Composite Travel Generative Adversarial Networks for Tabular and Sequential Population Synthesis](https://mlsanity.com/paper/2004.06838/) (2020)
- [Generative Models for Simulating Mobility Trajectories](https://mlsanity.com/paper/1811.12801/) (2018)
- [Copula-based transferable models for synthetic population generation](https://mlsanity.com/paper/2302.09193/) (2023)
- [Complexity-aware Large Scale Origin-Destination Network Generation via Diffusion Model](https://mlsanity.com/paper/2306.04873/) (2023)
- [A Large Language Model for Feasible and Diverse Population Synthesis](https://mlsanity.com/paper/2505.04196/) (2025)
- [Origin-Destination Network Generation via Gravity-Guided GAN](https://mlsanity.com/paper/2306.03390/) (2023)
- [Noise Matters: Diffusion Model-based Urban Mobility Generation with Collaborative Noise Priors](https://mlsanity.com/paper/2412.05000/) (2024)
- [Generating synthetic mobility data for a realistic population with RNNs to improve utility and privacy](https://mlsanity.com/paper/2201.01139/) (2022)
- [A Differentially Private Multi-Output Deep Generative Networks Approach For Activity Diary Synthesis](https://mlsanity.com/paper/2012.14574/) (2020)
- [Calibrating Path Choices and Train Capacities for Urban Rail Transit Simulation Models Using Smart Card and Train Movement Data](https://mlsanity.com/paper/2012.07731/) (2020)
- [Holdout-Based Fidelity and Privacy Assessment of Mixed-Type Synthetic Data](https://mlsanity.com/paper/2104.00635/) (2021)

## Abstract (arXiv)

Synthetic data offers a promising solution to the privacy and accessibility challenges of using smart card data in public transport research. Despite rapid progress in generative modeling, there is limited attention to comprehensive evaluation, leaving unclear how reliable, safe, and useful synthetic data truly are. Existing evaluations remain fragmented, typically limited to population-level representativeness or record-level privacy, without considering group-level variations or task-specific utility. To address this gap, we propose a Representativeness-Privacy-Utility (RPU) framework that systematically evaluates synthetic trip data across three complementary dimensions and three hierarchical levels (record, group, population). The framework integrates a consistent set of metrics to quantify similarity, disclosure risk, and practical usefulness, enabling transparent and balanced assessment of synthetic data quality. We apply the framework to benchmark twelve representative generation methods, spanning conventional statistical models, deep generative networks, and privacy-enhanced variants. Results show that synthetic data do not inherently guarantee privacy and there is no "one-size-fits-all" model, the trade-off between privacy and representativeness/utility is obvious. Conditional Tabular generative adversarial network (CTGAN) provide the most balanced trade-off and is suggested for practical applications. The RPU framework provides a systematic and reproducible basis for researchers and practitioners to compare synthetic data generation techniques and select appropriate methods in public transport applications.