mlsanityData to 7 Oct 2026

UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Web

arXiv 2310.18340 ↗First posted 22 Oct 2023cs.CL cs.AIcited by 77

Not on the map. Why?

Abstract (arXiv)

Urban region profiling from web-sourced data is of utmost importance for urban planning and sustainable development. We are witnessing a rising trend of LLMs for various fields, especially dealing with multi-modal data research such as vision-language learning, where the text modality serves as a supplement information for the image. Since textual modality has never been introduced into modality combinations in urban region profiling, we aim to answer two fundamental questions in this paper: i) Can textual modality enhance urban region profiling? ii) and if so, in what ways and with regard to which aspects? To answer the questions, we leverage the power of Large Language Models (LLMs) and introduce the first-ever LLM-enhanced framework that integrates the knowledge of textual modality into urban imagery profiling, named LLM-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining (UrbanCLIP). Specifically, it first generates a detailed textual description for each satellite image by an open-source Image-to-Text LLM. Then, the model is trained on the image-text pairs, seamlessly unifying natural language supervision for urban visual representation learning, jointly with contrastive loss and language modeling loss. Results on predicting three urban indicators in four major Chinese metropolises demonstrate its superior performance, with an average improvement of 6.1% on R^2 compared to the state-of-the-art methods. Our code and the image-language dataset will be released upon paper notification.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). Builds on 14 of its 32 references that are in the map. First posted on arXiv 22 Oct 2023.

Builds on

14 of 32 references.

References · 14

  1. 2019How Can We Know What Language Models Know?cited by 953
  2. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  3. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 402
  4. 2022Zero-Shot Video Question Answering via Frozen Bidirectional Language Modelscited by 229
  5. 2018Tile2Vec: Unsupervised representation learning for spatially distributed datacited by 109
  6. 2020Using satellite imagery to understand and promote sustainable developmentcited by 71
  7. 2020Urban2Vec: Incorporating Street View Imagery and POIs for Multi-Modal Urban Neighborhood Embeddingcited by 42
  8. 2022ERNIE-GeoL: A Geography-and-Language Pre-trained Model and its Applications in Baidu Mapscited by 25
  9. 2023Knowledge-infused Contrastive Learning for Urban Imagery-based Socioeconomic Predictioncited by 23
  10. 2018Take a Look Around: Using Street View and Satellite Images to Estimate House Pricescited by 16
  11. 2019Lightweight and Robust Representation of Economic Scales from Satellite Imagerycited by 15
  12. 2022Mutual Distillation Learning Network for Trajectory-User Linkingcited by 15
2 more
  1. 2023MGeo: Multi-Modal Geographic Pre-Training Methodcited by 12
  2. 2022Learning Economic Indicators by Aggregating Multi-Level Geospatial Informationcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 19; 2025: 41; 2026: 15.

77 papers.

Sort:

By area:

  1. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  2. 2025LLM Agents for Education: Advances and Applicationscited by 59
  3. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  4. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  5. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
  6. 2025SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearningcited by 31
  7. 2025CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Spacecited by 31
  8. 2024FineCLIPER: Multi-modal Fine-grained CLIP for Dynamic Facial Expression Recognition with AdaptERscited by 30
  9. 2024Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Modelscited by 30
  10. 2025MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Modelscited by 29
  11. 2024UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarioscited by 28
  12. 2025Recognition through Reasoning: Reinforcing Image Geo-localization with Large Vision-Language Modelscited by 27
38 more
  1. 2024CityBench: Evaluating the Capabilities of Large Language Models for Urban Taskscited by 26
  2. 2025MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detectioncited by 24
  3. 2024Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Modelscited by 23
  4. 2025Foundation Models for Spatio-Temporal Data Science: A Tutorial and Surveycited by 23
  5. 2025How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlookcited by 20
  6. 2024From Text to Transformation: A Comprehensive Review of Large Language Models' Versatilitycited by 19
  7. 2024UrbanVLP: Multi-Granularity Vision-Language Pretraining for Urban Socioeconomic Indicator Predictioncited by 19
  8. 2025DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruningcited by 19
  9. 2024BIGCity: A Universal Spatiotemporal Model for Unified Trajectory and Traffic State Data Analysiscited by 16
  10. 2023On the Promises and Challenges of Multimodal Foundation Models for Geographical, Environmental, Agricultural, and Urban Planning Applicationscited by 15
  11. 2025VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Modelscited by 14
  12. 2025FlexiReg: Flexible Urban Region Representation Learningcited by 13
  13. 2024GraphTeam: Facilitating Large Language Model-based Graph Analysis via Multi-Agent Collaborationcited by 12
  14. 2025A Survey of Large Language Model-Powered Spatial Intelligence Across Scales: Advances in Embodied Agents, Smart Cities, and Earth Sciencecited by 11
  15. 2024GeoReasoner: Reasoning On Geospatially Grounded Context For Natural Language Understandingcited by 10
  16. 2025POI-Enhancer: An LLM-based Semantic Enhancement Framework for POI Representation Learningcited by 10
  17. 2025Temporal Regularization Makes Your Video Generator Strongercited by 10
  18. 2025Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Reviewcited by 10
  19. 2024ST-LoRA: Low-rank Adaptation for Spatio-Temporal Forecastingcited by 9
  20. 2024Urban Region Pre-training and Prompting: A Graph-based Approachcited by 8
  21. 2024Self-Supervised Representation Learning for Geospatial Objects: A Surveycited by 8
  22. 2026Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Frameworkcited by 8
  23. 2023GeoSAM: Fine-tuning SAM with Multi-Modal Prompts for Mobility Infrastructure Segmentationcited by 7
  24. 2024MINER: Mining the Underlying Pattern of Modality-Specific Neurons in Multimodal Large Language Modelscited by 7
  25. 2025EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Modelscited by 7
  26. 2025CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoringcited by 7
  27. 2025CityLens: Evaluating Large Vision-Language Models for Urban Socioeconomic Sensingcited by 7
  28. 2025UrbanFeel: A Comprehensive Benchmark for Temporal and Perceptual Understanding of City Scenes through Human Perspectivecited by 7
  29. 2025Enriching Location Representation with Detailed Semantic Informationcited by 6
  30. 2024Space-aware Socioeconomic Indicator Inference with Heterogeneous Graphscited by 5
  31. 2025MoRA: Mobility as the Backbone for Geospatial Representation Learning at Scalecited by 4
  32. 2025Large Language Model Powered Intelligent Urban Agents: Concepts, Capabilities, and Applicationscited by 4
  33. 2025Reimagining Urban Science: Scaling Causal Inference with Large Language Modelscited by 3
  34. 2025Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysiscited by 3
  35. 2025From Points to Places: Towards Human Mobility-Driven Spatiotemporal Foundation Models via Understanding Placescited by 3
  36. 2025Mitigating Geospatial Knowledge Hallucination in Large Language Models: Benchmarking and Dynamic Factuality Aligningcited by 3
  37. 2025Urban-R1: Reinforced MLLMs Mitigate Geospatial Biases for Urban General Intelligencecited by 3
  38. 2024Beyond Generation: Unlocking Universal Editing via Self-Supervised Fine-Tuningcited by 2

50 most cited of 77 All 77 (JSON)