SpatialLLM: From Multi-modality Data to Urban Spatial Intelligence
Not on the map. Why?
Abstract (arXiv)
We propose SpatialLLM, a novel approach advancing spatial intelligence tasks in complex urban scenes. Unlike previous methods requiring geographic analysis tools or domain expertise, SpatialLLM is a unified language model directly addressing various spatial intelligence tasks without any training, fine-tuning, or expert intervention. The core of SpatialLLM lies in constructing detailed and structured scene descriptions from raw spatial data to prompt pre-trained LLMs for scene-based analysis. Extensive experiments show that, with our designs, pretrained LLMs can accurately perceive spatial distribution information and enable zero-shot execution of advanced spatial intelligence tasks, including urban planning, ecological analysis, traffic management, etc. We argue that multi-field knowledge, context length, and reasoning ability are key factors influencing LLM performances in urban analysis. We hope that SpatialLLM will provide a novel viable perspective for urban intelligent analysis and management. The code and dataset are available at https://github.com/WHU-USI3DV/SpatialLLM.
Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 32 of its 59 references that are in the map. First posted on arXiv 19 May 2025.
Builds on
32 of 59 references.
References · 32
- 2023GPT-4 Technical Reportcited by 18,396
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023Qwen Technical Reportcited by 3,344
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
- 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
- 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
- 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 365
- 2023An Embodied Generalist Agent in 3D Worldcited by 365
- 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
20 more
- 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
- 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planningcited by 210
- 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 204
- 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 175
- 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
- 2022Stratified Transformer for 3D Point Cloud Segmentationcited by 167
- 2023Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenescited by 129
- 2023On the Opportunities and Challenges of Foundation Models for Geospatial Artificial Intelligencecited by 88
- 2023Autonomous GIS: the next-generation AI-powered GIScited by 53
- 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 43
- 2023GeoGPT: Understanding and Processing Geospatial Tasks through An Autonomous GPTcited by 41
- 2024MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotationscited by 40
- 2024CityGPT: Empowering Urban Spatial Cognition of Large Language Modelscited by 30
- 2023UrbanBIS: a Large-scale Benchmark for Fine-grained Urban Building Instance Segmentationcited by 24
- 2023CityRefer: Geography-aware 3D Visual Grounding Dataset on City-scale Point Cloud Datacited by 18
- 2023Recent Advances in Multi-modal 3D Intelligence: A Comprehensive Survey and Evaluationcited by 12
- 2024SceneGPT: A Language Model for 3D Scene Understandingcited by 12
- 2024Tag Map: A Text-Based Map for Spatial Reasoning and Navigation with Large Language Modelscited by 12
- 20243D Question Answering for City Scene Understandingcited by 10
- 2024GeoReasoner: Reasoning On Geospatially Grounded Context For Natural Language Understandingcited by 10
Built on by
Citing papers per year: 2025: 2; 2026: 6.
8 papers.
- 2025GTR-Bench: Evaluating Geo-Temporal Reasoning in Vision-Language Modelscited by 1
- 2025RoadBench: Benchmarking MLLMs on Fine-Grained Spatial Understanding and Reasoning under Urban Road Scenarioscited by 0
- 2026Spatial-Morphological Modeling for Multi-Attribute Imputation of Urban Blockscited by 0
- 2026Evaluating Few-Shot Temporal Reasoning of LLMs for Human Activity Prediction in Smart Environmentscited by 0
- 2026Ground-Truth Depth in Vision Language Models: Spatial Context Understanding in Conversational AI for XR-Robotic Support in Emergency First Responsecited by 0
- 2026UrbanWell: Benchmarking Multimodal Large Language Models for Spatio-Temporal Urban Wellbeing Analyticscited by 0
- 2026LifePlanner: Evaluating LLM Agents for Geo-spatial Planning with Social Media Datacited by 0
- 2026CitySTAR: Structured and Topology-Aware Reasoning for Open-Vocabulary Urban 3D Groundingcited by 0
- 2026CitySTAR: Structured and Topology-Aware Reasoning for Open-Vocabulary Urban 3D Grounding17 Sep 2026
- 2026LifePlanner: Evaluating LLM Agents for Geo-spatial Planning with Social Media Data25 Aug 2026
- 2026UrbanWell: Benchmarking Multimodal Large Language Models for Spatio-Temporal Urban Wellbeing Analytics14 Jun 2026
- 2026Ground-Truth Depth in Vision Language Models: Spatial Context Understanding in Conversational AI for XR-Robotic Support in Emergency First Response16 Feb 2026
- 2026Spatial-Morphological Modeling for Multi-Attribute Imputation of Urban Blocks11 Feb 2026
- 2026Evaluating Few-Shot Temporal Reasoning of LLMs for Human Activity Prediction in Smart Environments20 Jan 2026
- 2025RoadBench: Benchmarking MLLMs on Fine-Grained Spatial Understanding and Reasoning under Urban Road Scenarios22 Nov 2025
- 2025GTR-Bench: Evaluating Geo-Temporal Reasoning in Vision-Language Models9 Oct 2025
Similar, not cited
Close in wording, no citation either way
- 2025Can Large Language Models Integrate Spatial Data? Empirical Insights into Reasoning Strengths and Computational Weaknesses
- 2025A Survey of Large Language Model-Powered Spatial Intelligence Across Scales: Advances in Embodied Agents, Smart Cities, and Earth Science
- 2025UrbanLLaVA: A Multi-modal Large Language Model for Urban Intelligence with Spatial Reasoning and Understanding
- 2025SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence
- 2024StreetviewLLM: Extracting Geographic Information Using a Chain-of-Thought Multimodal Large Language Model
- 2026Can LLMs See Without Pixels? Benchmarking Spatial Intelligence from Textual Descriptions
- 2026Mind the Spatial Gap: A Benchmark for Geospatial Reasoning and Constrained Spatial Editing
- 2025USTBench: Benchmarking and Dissecting Spatiotemporal Reasoning of LLMs as Urban Agents