Towards General Urban Monitoring with Vision-Language Models: A Review, Evaluation, and a Research Agenda
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
Urban monitoring of public infrastructure (such as waste bins, road signs, vegetation, sidewalks, and construction sites) poses significant challenges due to the diversity of objects, environments, and contextual conditions involved. Current state-of-the-art approaches typically rely on a combination of IoT sensors and manual inspections, which are costly, difficult to scale, and often misaligned with citizens' perception formed through direct visual observation. This raises a critical question: Can machines now "see" like citizens and infer informed opinions about the condition of urban infrastructure? Vision-Language Models (VLMs), which integrate visual understanding with natural language reasoning, have recently demonstrated impressive capabilities in processing complex visual information, turning them into a promising technology to address this challenge. This systematic review investigates the role of VLMs in urban monitoring, with particular emphasis on zero-shot applications. Following the PRISMA methodology, we analyzed 32 peer-reviewed studies published between 2021 and 2025 to address four core research questions: (1) What urban monitoring tasks have been effectively addressed using VLMs? (2) Which VLM architectures and frameworks are most commonly used and demonstrate superior performance? (3) What datasets and resources support this emerging field? (4) How are VLM-based applications evaluated, and what performance levels have been reported?
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 63 references that are in the map. First posted on arXiv 14 Oct 2025.
Builds on
31 of 63 references.
Closest ancestors · 5
Same direction, newest first
- 2025OpenCity3D: What do Vision-Language Models know about Urban Environments?21 Mar
- 2024UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarios30 Aug
- 2024CityBench: Evaluating the Capabilities of Large Language Models for Urban Tasks20 Jun
- 2024GeoReasoner: Geo-localization with Reasoning in Street Views using a Large Vision-Language Model3 Jun
- 2023Vision-Language Models in Remote Sensing: Current Progress and Future Trends9 May
Other references · 26
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
- 2024Depth Anything V2cited by 1,385
- 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
- 2024TinyLlama: An Open-Source Small Language Modelcited by 604
- 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
- 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 498
14 more
- 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 403
- 2020VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrievalcited by 188
- 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 124
- 2024Foundation Models for Video Understanding: A Surveycited by 46
- 2023ViPlanner: Visual Semantic Imperative Learning for Local Navigationcited by 40
- 2020Adaptive-Attentive Geolocalization from few queries: a hybrid approachcited by 39
- 2019EvalAI: Towards Better Evaluation Systems for AI Agentscited by 34
- 2024CityLLaVA: Efficient Fine-Tuning for VLMs in City Scenariocited by 27
- 2023Kick Back & Relax: Learning to Reconstruct the World by Watching SlowTVcited by 21
- 2023VLPD: Context-Aware Pedestrian Detection via Vision-Language Semantic Self-Supervisioncited by 19
- 2023CLIPTER: Looking at the Bigger Picture in Scene Text Recognitioncited by 16
- 2023Real-time Multi-Class Helmet Violation Detection Using Few-Shot Data Sampling Technique and YOLOv8cited by 13
- 2022A Priority Map for Vision-and-Language Navigation with Trajectory Plans and Feature-Location Cuescited by 5
- 2024Hierarchical end-to-end autonomous navigation through few-shot waypoint detectioncited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2024Towards Zero-Shot Annotation of the Built Environment with Vision-Language Models (Vision Paper)
- 2025InfraGPT Smart Infrastructure: An End-to-End VLM-Based Framework for Detecting and Managing Urban Defects
- 2026Built Environment Reasoning from Remote Sensing Imagery Using Large Vision--Language Models
- 2026Unleashing the Capabilities of Large Vision-Language Models for Intelligent Perception of Roadside Infrastructure
- 2026AVA-VLM: Adaptive Visual Attention-Vision Language Model for In-the-Wild Construction Site Monitoring
- 2024Advancements in Visual Language Models for Remote Sensing: Datasets, Capabilities, and Enhancement Techniques
- 2025How Well Do Vision--Language Models Understand Cities? A Comparative Study on Spatial Reasoning from Street-View Images
- 2025Can a Large Language Model Assess Urban Design Quality? Evaluating Walkability Metrics Across Expertise Levels