mlsanityData to 7 Oct 2026

CityRefer: Geography-aware 3D Visual Grounding Dataset on City-scale Point Cloud Data

arXiv 2310.18773 ↗First posted 28 Oct 2023cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

City-scale 3D point cloud is a promising way to express detailed and complicated outdoor structures. It encompasses both the appearance and geometry features of segmented city components, including cars, streets, and buildings, that can be utilized for attractive applications such as user-interactive navigation of autonomous vehicles and drones. However, compared to the extensive text annotations available for images and indoor scenes, the scarcity of text annotations for outdoor scenes poses a significant challenge for achieving these applications. To tackle this problem, we introduce the CityRefer dataset for city-level visual grounding. The dataset consists of 35k natural language descriptions of 3D objects appearing in SensatUrban city scenes and 5k landmarks labels synchronizing with OpenStreetMap. To ensure the quality and accuracy of the dataset, all descriptions and labels in the CityRefer dataset are manually verified. We also have developed a baseline system that can learn encoded language descriptions, 3D object instances, and geographical information about the city's landmarks to perform visual grounding on the CityRefer dataset. To the best of our knowledge, the CityRefer dataset is the largest city-level visual grounding dataset for localizing specific 3D objects.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 45 of its 64 references that are in the map. First posted on arXiv 28 Oct 2023.

Builds on

45 of 64 references.

References · 45

  1. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  2. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  3. 2018Datasheets for Datasetscited by 1,371
  4. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  5. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  6. 2016Modeling Context in Referring Expressionscited by 1,054
  7. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  8. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  9. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  10. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  11. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  12. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 446
33 more
  1. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  2. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 404
  3. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  4. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  5. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  6. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  7. 2020Scan2Cap: Context-aware Dense Captioning in RGB-D Scanscited by 215
  8. 2022SoftGroup for 3D Instance Segmentation on Point Cloudscited by 197
  9. 2019RIO: 3D Object Instance Re-Localization in Changing Indoor Environmentscited by 187
  10. 2017Semantic3D.net: A new Large-scale Point Cloud Classification Benchmarkcited by 158
  11. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 140
  12. 2022Multi-View Transformer for 3D Visual Groundingcited by 138
  13. 2021Bottom Up Top Down Detection Transformers for Language Grounding in Images and Point Cloudscited by 128
  14. 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
  15. 2022RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Datacited by 121
  16. 2022EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Groundingcited by 98
  17. 2021LanguageRefer: Spatial-Language Model for 3D Visual Groundingcited by 88
  18. 2021TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Groundingcited by 86
  19. 2020Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadwayscited by 77
  20. 2022X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioningcited by 70
  21. 2017Paris-Lille-3D: a large and high-quality ground truth urban point cloud dataset for automatic segmentation and classificationcited by 65
  22. 2020DALES: A Large-scale Aerial LiDAR Data Set for Semantic Segmentationcited by 65
  23. 2022STPLS3D: A Large-Scale Synthetic and Real Aerial Photogrammetry 3D Point Cloud Datasetcited by 52
  24. 2022SensatUrban: Learning Semantics from Urban-Scale Photogrammetric Point Cloudscited by 49
  25. 2022Text2Pos: Text-to-Point-Cloud Cross-Modal Localizationcited by 40
  26. 2022Spatiality-guided Transformer for 3D Dense Captioning on Point Cloudscited by 40
  27. 2015Semantic Instance Annotation of Street Scenes by 3D to 2D Label Transfercited by 39
  28. 2021Refer-it-in-RGBD: A Bottom-up Approach for 3D Visual Grounding in RGBD Imagescited by 33
  29. 2022ScanEnts3D: Exploiting Phrase-to-3D-Object Correspondences for Improved Visio-Linguistic Models in 3D Scenescited by 32
  30. 2019DublinCity: Annotated LiDAR Point Cloud and its Applicationscited by 31
  31. 2020Campus3D: A Photogrammetry Point Cloud Benchmark for Hierarchical Understanding of Outdoor Scenecited by 27
  32. 2022Towards Explainable 3D Grounded Visual Question Answering: A New Benchmark and Strong Baselinecited by 25
  33. 2022Toward Explainable and Fine-Grained 3D Grounding through Referring Textual Phrasescited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 8; 2026: 6.

15 papers.

Sort:

By area:

  1. 2025Openfly: A comprehensive platform for aerial vision-language navigationcited by 43
  2. 2024CityNav: A Large-Scale Dataset for Real-World Aerial Navigationcited by 40
  3. 2025PointArena: Probing Multimodal Grounding Through Language-Guided Pointingcited by 30
  4. 2025Aerial Vision-and-Language Navigation with Grid-based View Selection and Map Constructioncited by 12
  5. 2025SpatialLLM: From Multi-modality Data to Urban Spatial Intelligencecited by 8
  6. 20253EED: Ground Everything Everywhere in 3Dcited by 8
  7. 2025ViGiL3D: A Linguistically Diverse Dataset for 3D Visual Groundingcited by 7
  8. 2025Talk2Event: Grounded Understanding of Dynamic Scenes from Event Camerascited by 6
  9. 2026AirNav: A Large-Scale UAV Vision-and-Language Navigation Dataset with Natural and Diverse Instructionscited by 5
  10. 2026VLM-Loc: Localization in Point Cloud Maps via Vision-Language Modelscited by 4
  11. 2026ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigationcited by 2
  12. 2026HTNav: A Hybrid Navigation Framework with Tiered Structure for Urban Aerial Vision-and-Language Navigationcited by 2
3 more
  1. 2025Visual Grounding from Event Camerascited by 1
  2. 2026Stitch4D: Sparse Multi-Location 4D Urban Reconstruction via Spatio-Temporal Interpolationcited by 0
  3. 2026Structured World-State Reasoning for Agentic Robotic Searchcited by 0