UrbanTwin: Building High-Fidelity Digital Twins for Sim2Real LiDAR Perception and Evaluation
Not on the map. Why?
Abstract (arXiv)
LiDAR-based perception in intelligent transportation systems (ITS) relies on deep neural networks trained with large-scale labeled datasets. However, creating such datasets is expensive, time-consuming, and labor-intensive, limiting the scalability of perception systems. Sim2Real learning offers a scalable alternative, but its success depends on the simulation's fidelity to real-world environments, dynamics, and sensors. This tutorial introduces a reproducible workflow for building high-fidelity digital twins (HiFi DTs) to generate realistic synthetic datasets. We outline practical steps for modeling static geometry, road infrastructure, and dynamic traffic using open-source resources such as satellite imagery, OpenStreetMap, and sensor specifications. The resulting environments support scalable and cost-effective data generation for robust Sim2Real learning. Using this workflow, we have released three synthetic LiDAR datasets, namely UT-LUMPI, UT-V2X-Real, and UT-TUMTraf-I, which closely replicate real locations and outperform real-data-trained baselines in perception tasks. This guide enables broader adoption of HiFi DTs in ITS research and deployment.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 39 references that are in the map. First posted on arXiv 3 Sep 2025.
Builds on
12 of 39 references.
References · 12
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
- 2022DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detectioncited by 273
- 2020LiDARsim: Realistic LiDAR Simulation by Leveraging the Real Worldcited by 137
- 2021VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehiclescited by 81
- 2024V2X-Real: a Large-Scale Dataset for Vehicle-to-Everything Cooperative Perceptioncited by 66
- 2020ePointDA: An End-to-End Simulation-to-Real Domain Adaptation Framework for LiDAR Point Cloud Segmentationcited by 56
- 2019How much real data do we actually need: Analyzing object detection performance using synthetic and real datacited by 29
- 2021IPS300+: a Challenging Multimodal Dataset for Intersection Perception Systemcited by 26
- 2023Choose Your Simulator Wisely: A Review on Open-source Simulators for Autonomous Drivingcited by 26
- 2021BAAI-VANJEE Roadside Dataset: Towards the Connected Automated Vehicle Highway technologies in Challenging Environments of Chinacited by 19
Built on by
Not cited yet.