mlsanityData to 14 Oct 2026

Urbex: Agentic Spatial Grounding for City-Scale 3D Scenes

arXiv 2610.18356 ↗First posted 13 Oct 2026cs.CV cs.AIcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

City-scale 3D grounding is commonly formulated as candidate ranking, which requires precomputed instance proposals and becomes inefficient in large outdoor scenes. We propose Urbex, an agentic vision-language framework that instead casts grounding as active spatial search. Urbex represents a city-scale 3D scene as an interactive multi-view environment, allowing a VLM agent to query landmarks, zoom into local regions, render oblique views, and commit a grounded bounding box. We optimize this tool-use policy with reinforcement learning on CityRefer, using smooth localization rewards and lightweight shaping rewards for efficient exploration. Experiments show that Urbex achieves the best Hit@1 and IoU on CityRefer, supports candidate-free grounding without ground-truth boxes, and is substantially faster than the strongest city-scale ranking baseline. Zero-shot results on STPLS3D-Refer further indicate improved cross-scene robustness.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 42 of its 48 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

42 of 48 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Think, Act, Build: An Agentic Framework with Vision Language Models for Zero-Shot 3D Visual Grounding1 Apr
  2. 2026Thinking with Map: Reinforced Parallel Map-Augmented Agent for Geolocalization8 Jan
  3. 2023An Embodied Generalist Agent in 3D World18 Nov

Other references · 39

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  6. 2023Let's Verify Step by Stepcited by 3,789
  7. 2025Qwen3-VL Technical Reportcited by 2,793
  8. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  9. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  10. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 603
  11. 2022Visual Programming: Compositional visual reasoning without trainingcited by 516
  12. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
27 more
  1. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 429
  2. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 365
  3. 2020PointGroup: Dual-Set Point Grouping for 3D Instance Segmentationcited by 321
  4. 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
  5. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 239
  6. 2022SoftGroup for 3D Instance Segmentation on Point Cloudscited by 200
  7. 2024MobileVLM V2: Faster and Stronger Baseline for Vision Language Modelcited by 174
  8. 2025GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoningcited by 150
  9. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 141
  10. 2022Multi-View Transformer for 3D Visual Groundingcited by 140
  11. 2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 122
  12. 20223D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selectioncited by 110
  13. 2022EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Groundingcited by 99
  14. 2021LanguageRefer: Spatial-Language Model for 3D Visual Groundingcited by 89
  15. 2026SkillClaw: Let Skills Evolve Collectively with Agentic Evolvercited by 81
  16. 2022STPLS3D: A Large-Scale Synthetic and Real Aerial Photogrammetry 3D Point Cloud Datasetcited by 53
  17. 2022SensatUrban: Learning Semantics from Urban-Scale Photogrammetric Point Cloudscited by 50
  18. 2026Code2World: A GUI World Model via Renderable Code Generationcited by 29
  19. 2025Video-STAR: Reinforcing Open-Vocabulary Action Recognition with Toolscited by 26
  20. 2024A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directionscited by 25
  21. 2023Language-Guided 3D Object Detection in Point Cloud for Autonomous Drivingcited by 20
  22. 2023CityRefer: Geography-aware 3D Visual Grounding Dataset on City-scale Point Cloud Datacited by 18
  23. 20253EED: Ground Everything Everywhere in 3Dcited by 12
  24. 2025View-on-Graph: Zero-shot 3D Visual Grounding via Vision-Language Reasoning on Scene Graphscited by 4
  25. 2026Urban Socio-Semantic Segmentation with Vision-Language Reasoningcited by 3
  26. 2026LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineeringcited by 3
  27. 2026UniGround: Universal 3D Visual Grounding via Training-Free Scene Parsingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026AgentGrounder: Zero-Shot 3D Visual Pointcloud Grounding using Multimodal Language Models
  2. 2026UrbanGround: From Local Perception to Spatial Agency in a Real-Scale City
  3. 2026Finder: Agentic Closed-Loop Object Finding for Embodied Grounding
  4. 2025SPAZER: Spatial-Semantic Progressive Reasoning Agent for Zero-shot 3D Visual Grounding
  5. 2026Scaling Diverse Language Generation for 3D Visual Grounding
  6. 2026Meanings and Measurements: Multi-Agent Probabilistic Grounding for Vision-Language Navigation
  7. 2025Zero-Shot 3D Visual Grounding from Vision-Language Models
  8. 2026SpotAgent: Grounding Visual Geo-localization in Large Vision-Language Models through Agentic Reasoning