mlsanityData to 14 Oct 2026

GeoChat: Grounded Large Vision-Language Model for Remote Sensing

arXiv 2311.15826 ↗First posted 24 Nov 2023cs.CV cs.AIcited by 328

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Recent advancements in Large Vision-Language Models (VLMs) have shown great promise in natural image domains, allowing users to hold a dialogue about given visual content. However, such general-domain VLMs perform poorly for Remote Sensing (RS) scenarios, leading to inaccurate or fabricated information when presented with RS domain-specific queries. Such a behavior emerges due to the unique challenges introduced by RS imagery. For example, to handle high-resolution RS imagery with diverse scale changes across categories and many small objects, region-level reasoning is necessary alongside holistic scene interpretation. Furthermore, the lack of domain-specific multimodal instruction following data as well as strong backbone models for RS make it hard for the models to align their behavior with user queries. To address these limitations, we propose GeoChat - the first versatile remote sensing VLM that offers multitask conversational capabilities with high-resolution RS images. Specifically, GeoChat can not only answer image-level queries but also accepts region inputs to hold region-specific dialogue. Furthermore, it can visually ground objects in its responses by referring to their spatial coordinates. To address the lack of domain-specific datasets, we generate a novel RS multimodal instruction-following dataset by extending image-text pairs from existing diverse RS datasets. We establish a comprehensive benchmark for RS multitask conversations and compare with a number of baseline methods. GeoChat demonstrates robust zero-shot performance on various RS tasks, e.g., image and region captioning, visual question answering, scene classification, visually grounded conversations and referring detection. Our code is available at https://github.com/mbzuai-oryx/geochat.

Key facts. Cited by 328 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 40 references that are in the map. First posted on arXiv 24 Nov 2023.

Builds on

29 of 40 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023RSGPT: A Remote Sensing Vision Language Model and Benchmark28 Jul

Other references · 28

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  8. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  9. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  10. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  11. 2016Modeling Context in Referring Expressionscited by 1,063
  12. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
16 more
  1. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 797
  2. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  3. 2017DOTA: A Large-scale Dataset for Object Detection in Aerial Imagescited by 623
  4. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 550
  5. 2016AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classificationcited by 347
  6. 2021LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentationcited by 261
  7. 2020RSVQA: Visual Question Answering for Remote Sensing Datacited by 203
  8. 2020FloodNet: A High Resolution Aerial Imagery Dataset for Post Flood Scene Understandingcited by 181
  9. 2022Advancing Plain Vision Transformer Towards Remote Sensing Foundation Modelcited by 149
  10. 2021FAIR1M: A Benchmark Dataset for Fine-grained Object Recognition in High-Resolution Remote Sensing Imagerycited by 143
  11. 2022RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Datacited by 123
  12. 2021Anchor-free Oriented Proposal Generator for Object Detectioncited by 105
  13. 2023XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Modelscited by 61
  14. 2022From Easy to Hard: Learning Language-guided Curriculum for Visual Question Answering on Remote Sensing Datacited by 39
  15. 2023SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Modelcited by 38
  16. 2017Learning to Rank Question Answer Pairs with Holographic Dual LSTM Architecturecited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 59; 2025: 122; 2026: 147.

328 papers.

Sort:

By area:

  1. 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domaincited by 184
  2. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLMcited by 184
  3. 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Modelcited by 138
  4. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 135
  5. 2024VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understandingcited by 121
  6. 2024SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understandingcited by 104
  7. 2024Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Wantcited by 101
  8. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
  9. 2024EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialoguescited by 85
  10. 2025PyVision: Agentic Vision with Dynamic Toolingcited by 79
  11. 2024HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Modelcited by 73
  12. 2024TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Datacited by 70
38 more
  1. 2024LLaRA: Supercharging Robot Learning Data for Vision-Language Policycited by 52
  2. 2024GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Taskscited by 51
  3. 2025XLRS-Bench: Could Your Multimodal LLMs Understand Extremely Large Ultra-High-Resolution Remote Sensing Imagery?cited by 51
  4. 2024Locate Anything on Earth: Advancing Open-Vocabulary Object Detection for Remote Sensing Communitycited by 50
  5. 2024RS-Agent: Automating Remote Sensing Tasks through Intelligent Agentcited by 49
  6. 2024LSKNet: A Foundation Lightweight Backbone for Remote Sensingcited by 44
  7. 2025GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolutioncited by 44
  8. 2024MetaEarth: A Generative Foundation Model for Global-Scale Remote Sensing Image Generationcited by 42
  9. 2024Good at captioning, bad at counting: Benchmarking GPT-4V on Earth observation datacited by 39
  10. 2025RemoteSAM: Towards Segment Anything for Earth Observationcited by 39
  11. 2024Towards Vision-Language Geo-Foundation Model: A Surveycited by 36
  12. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 36
  13. 2025DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Responsecited by 36
  14. 2025When Large Vision-Language Model Meets Large Remote Sensing Imagery: Coarse-to-Fine Text-Guided Token Pruningcited by 35
  15. 2024Deep Learning for Cross-Domain Data Fusion in Urban Computing: Taxonomy, Advances, and Outlookcited by 33
  16. 2024Large Language Models for Captioning and Retrieving Remote Sensing Imagescited by 30
  17. 2024EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensingcited by 29
  18. 2024UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarioscited by 28
  19. 2024CityBench: Evaluating the Capabilities of Large Language Models for Urban Taskscited by 27
  20. 2024CityCraft: A Real Crafter for 3D City Generationcited by 26
  21. 2024MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Modelcited by 26
  22. 2025TerraFM: A Scalable Foundation Model for Unified Multisensor Earth Observationcited by 25
  23. 2024ChatEarthNet: A Global-Scale Image-Text Dataset Empowering Vision-Language Geo-Foundation Modelscited by 24
  24. 2024Exploring Fine-Grained Image-Text Alignment for Referring Remote Sensing Image Segmentationcited by 24
  25. 2024AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuningcited by 24
  26. 2025FlexiMo: A Flexible Remote Sensing Foundation Modelcited by 24
  27. 2024CHOICE: Benchmarking the Remote Sensing Capabilities of Large Vision-Language Modelscited by 22
  28. 2025Multi-Agent Geospatial Copilots for Remote Sensing Workflowscited by 22
  29. 2025Multimodal Referring Segmentation: A Surveycited by 22
  30. 2024An LLM Agent for Automatic Geospatial Data Analysiscited by 21
  31. 2025RingMo-Agent: A Unified Remote Sensing Foundation Model for Multi-Platform and Multi-Modal Reasoningcited by 20
  32. 2025Geo-R1: Improving Few-Shot Geospatial Referring Expression Understanding with Reinforcement Fine-Tuningcited by 20
  33. 2025A Survey on Remote Sensing Foundation Models: From Vision to Multimodalitycited by 19
  34. 2025Vision-Language Modeling Meets Remote Sensing: Models, Datasets and Perspectivescited by 18
  35. 2025DynamicVL: Benchmarking Multimodal Large Language Models for Dynamic City Understandingcited by 18
  36. 2025EarthMind: Leveraging Cross-Sensor Data for Advanced Earth Observation Interpretation with a Unified Multimodal LLMcited by 18
  37. 2024Semantic-CC: Boosting Remote Sensing Image Change Captioning via Foundational Knowledge and Semantic Guidancecited by 17
  38. 2024CAMEL-Bench: A Comprehensive Arabic LMM Benchmarkcited by 17

50 most cited of 328 All 328 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025GeoRSMLLM: A Multimodal Large Language Model for Vision-Language Tasks in Geoscience and Remote Sensing
  2. 2025GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensing
  3. 2023RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing
  4. 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model
  5. 2023Vision-Language Models in Remote Sensing: Current Progress and Future Trends
  6. 2024From Pixels to Prose: Advancing Multi-Modal Language Models for Remote Sensing
  7. 2024GeoGround: A Unified Large Vision-Language Model for Remote Sensing Visual Grounding
  8. 2024Remote Sensing ChatGPT: Solving Remote Sensing Tasks with ChatGPT and Visual Models