# TUM2TWIN: Introducing the Large-Scale Multimodal Urban Digital Twin Benchmark Dataset

arXiv: https://arxiv.org/abs/2505.07396 · HTML: https://mlsanity.com/paper/2505.07396/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 86 references that are in the map.
- First posted on arXiv 12 May 2025.

## Builds on (most cited first, 25 of 25)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes](https://mlsanity.com/paper/2308.11417/) (2023)
- [Nerfstudio: A Modular Framework for Neural Radiance Field Development](https://mlsanity.com/paper/2302.04264/) (2023)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark](https://mlsanity.com/paper/1704.03847/) (2017)
- [Capturing, Reconstructing, and Simulating: the UrbanScene3D Dataset](https://mlsanity.com/paper/2107.04286/) (2021)
- [Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadways](https://mlsanity.com/paper/2003.08284/) (2020)
- [The Hessigheim 3D (H3D) Benchmark on Semantic Segmentation of High-Resolution 3D Point Clouds and Textured Meshes from UAV LiDAR and Multi-View-Stereo](https://mlsanity.com/paper/2102.05346/) (2021)
- [Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D Mapping](https://mlsanity.com/paper/2111.11348/) (2021)
- [Building3D: An Urban-Scale Dataset and Benchmarks for Learning Roof Structures from Point Clouds](https://mlsanity.com/paper/2307.11914/) (2023)
- [SUM: A Benchmark Dataset of Semantic Urban Meshes](https://mlsanity.com/paper/2103.00355/) (2021)
- [Scan2LoD3: Reconstructing semantic 3D building models at LoD3 using ray casting and Bayesian networks](https://mlsanity.com/paper/2305.06314/) (2023)
- [Virtual laser scanning with HELIOS++: A novel take on ray tracing-based simulation of topographic 3D laser scanning](https://mlsanity.com/paper/2101.09154/) (2021)
- [PolyGNN: Polyhedron-based Graph Neural Network for 3D Building Reconstruction from Point Clouds](https://mlsanity.com/paper/2307.08636/) (2023)
- [CityJSON: a compact and easy-to-use encoding of the CityGML data model](https://mlsanity.com/paper/1902.09155/) (2019)
- [Combining visibility analysis and deep learning for refinement of semantic 3D building models by conflict classification](https://mlsanity.com/paper/2303.05998/) (2023)
- [TUM-FAcCADE: Reviewing and enriching point cloud benchmarks for faccade segmentation](https://mlsanity.com/paper/2304.07140/) (2023)
- [Building-PCC: Building Point Cloud Completion Benchmarks](https://mlsanity.com/paper/2404.15644/) (2024)
- [MLS2LoD3: Refining low LoDs building models with MLS point clouds to reconstruct semantic LoD3 building models](https://mlsanity.com/paper/2402.06288/) (2024)
- [Classifying point clouds at the facade-level using geometric features and deep learning networks](https://mlsanity.com/paper/2402.06506/) (2024)
- [Analyzing the impact of semantic LoD3 building models on image-based vehicle localization](https://mlsanity.com/paper/2407.21432/) (2024)
- [FacaDiffy: Inpainting Unseen Facade Parts Using Diffusion Models](https://mlsanity.com/paper/2502.14940/) (2025)

## Built on by (most cited first, 5 of 5)

- [L2M-Reg: Building-level Uncertainty-aware Registration of Outdoor LiDAR Point Clouds and Semantic 3D City Models](https://mlsanity.com/paper/2509.16832/) (2025)
- [From Theory to Throughput: CUDA-Optimized APML for Large-Batch 3D Learning](https://mlsanity.com/paper/2512.19743/) (2025)
- [Radiometric fingerprinting of object surfaces using mobile laser scanning and semantic 3D road space models](https://mlsanity.com/paper/2603.11252/) (2026)
- [GS4City: Hierarchical Semantic Gaussian Splatting via City-Model Priors](https://mlsanity.com/paper/2604.11401/) (2026)
- [Accuracy potential of visual localization exploiting high-end street-level imagery](https://mlsanity.com/paper/2607.24409/) (2026)

## Abstract (arXiv)

Urban Digital Twins (UDTs) have become essential for managing cities and integrating complex, heterogeneous data from diverse sources. Creating UDTs involves challenges at multiple process stages, including acquiring accurate 3D source data, reconstructing high-fidelity 3D models, maintaining models' updates, and ensuring seamless interoperability to downstream tasks. Current datasets are usually limited to one part of the processing chain, hampering comprehensive UDTs validation. To address these challenges, we introduce the first comprehensive multimodal Urban Digital Twin benchmark dataset: TUM2TWIN. This dataset includes georeferenced, semantically aligned 3D models and networks along with various terrestrial, mobile, aerial, and satellite observations boasting 32 data subsets over roughly 100,000 m2 and currently 767 GB of data. By ensuring georeferenced indoor-outdoor acquisition, high accuracy, and multimodal data integration, the benchmark supports robust analysis of sensors and the development of advanced reconstruction methods. Additionally, we explore downstream tasks demonstrating the potential of TUM2TWIN, including novel view synthesis of NeRF and Gaussian Splatting, solar potential analysis, point cloud semantic segmentation, and LoD3 building reconstruction. We are convinced this contribution lays a foundation for overcoming current limitations in UDT creation, fostering new research directions and practical solutions for smarter, data-driven urban environments. The project is available under: https://tum2t.win