mlsanityData to 14 Oct 2026

TUM2TWIN: Introducing the Large-Scale Multimodal Urban Digital Twin Benchmark Dataset

arXiv 2505.07396 ↗First posted 12 May 2025cs.CV cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Urban Digital Twins (UDTs) have become essential for managing cities and integrating complex, heterogeneous data from diverse sources. Creating UDTs involves challenges at multiple process stages, including acquiring accurate 3D source data, reconstructing high-fidelity 3D models, maintaining models' updates, and ensuring seamless interoperability to downstream tasks. Current datasets are usually limited to one part of the processing chain, hampering comprehensive UDTs validation. To address these challenges, we introduce the first comprehensive multimodal Urban Digital Twin benchmark dataset: TUM2TWIN. This dataset includes georeferenced, semantically aligned 3D models and networks along with various terrestrial, mobile, aerial, and satellite observations boasting 32 data subsets over roughly 100,000 m2 and currently 767 GB of data. By ensuring georeferenced indoor-outdoor acquisition, high accuracy, and multimodal data integration, the benchmark supports robust analysis of sensors and the development of advanced reconstruction methods. Additionally, we explore downstream tasks demonstrating the potential of TUM2TWIN, including novel view synthesis of NeRF and Gaussian Splatting, solar potential analysis, point cloud semantic segmentation, and LoD3 building reconstruction. We are convinced this contribution lays a foundation for overcoming current limitations in UDT creation, fostering new research directions and practical solutions for smarter, data-driven urban environments. The project is available under: https://tum2t.win

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 25 of its 86 references that are in the map. First posted on arXiv 12 May 2025.

Builds on

25 of 86 references.

References · 25

  1. 2023Segment Anythingcited by 8,234
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  3. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  4. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  5. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
  6. 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 600
  7. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 575
  8. 2017Semantic3D.net: A new Large-scale Point Cloud Classification Benchmarkcited by 158
  9. 2021Capturing, Reconstructing, and Simulating: the UrbanScene3D Datasetcited by 118
  10. 2020Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadwayscited by 77
  11. 2021The Hessigheim 3D (H3D) Benchmark on Semantic Segmentation of High-Resolution 3D Point Clouds and Textured Meshes from UAV LiDAR and Multi-View-Stereocited by 37
  12. 2021Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D Mappingcited by 25
13 more
  1. 2023Building3D: An Urban-Scale Dataset and Benchmarks for Learning Roof Structures from Point Cloudscited by 24
  2. 2021SUM: A Benchmark Dataset of Semantic Urban Meshescited by 23
  3. 2023Scan2LoD3: Reconstructing semantic 3D building models at LoD3 using ray casting and Bayesian networkscited by 17
  4. 2021Virtual laser scanning with HELIOS++: A novel take on ray tracing-based simulation of topographic 3D laser scanningcited by 11
  5. 2023PolyGNN: Polyhedron-based Graph Neural Network for 3D Building Reconstruction from Point Cloudscited by 10
  6. 2019CityJSON: a compact and easy-to-use encoding of the CityGML data modelcited by 8
  7. 2023Combining visibility analysis and deep learning for refinement of semantic 3D building models by conflict classificationcited by 8
  8. 2023TUM-FAcCADE: Reviewing and enriching point cloud benchmarks for faccade segmentationcited by 5
  9. 2024Building-PCC: Building Point Cloud Completion Benchmarkscited by 5
  10. 2024MLS2LoD3: Refining low LoDs building models with MLS point clouds to reconstruct semantic LoD3 building modelscited by 4
  11. 2024Classifying point clouds at the facade-level using geometric features and deep learning networkscited by 3
  12. 2024Analyzing the impact of semantic LoD3 building models on image-based vehicle localizationcited by 3
  13. 2025FacaDiffy: Inpainting Unseen Facade Parts Using Diffusion Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025L2M-Reg: Building-level Uncertainty-aware Registration of Outdoor LiDAR Point Clouds and Semantic 3D City Modelscited by 1
  2. 2025From Theory to Throughput: CUDA-Optimized APML for Large-Batch 3D Learningcited by 0
  3. 2026Radiometric fingerprinting of object surfaces using mobile laser scanning and semantic 3D road space modelscited by 0
  4. 2026GS4City: Hierarchical Semantic Gaussian Splatting via City-Model Priorscited by 0
  5. 2026Accuracy potential of visual localization exploiting high-end street-level imagerycited by 0