mlsanityData to 7 Oct 2026

RSGPT: A Remote Sensing Vision Language Model and Benchmark

arXiv 2307.15266 ↗First posted 28 Jul 2023cs.CVcited by 204

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

The emergence of large-scale large language models, with GPT-4 as a prominent example, has significantly propelled the rapid advancement of artificial general intelligence and sparked the revolution of Artificial Intelligence 2.0. In the realm of remote sensing (RS), there is a growing interest in developing large vision language models (VLMs) specifically tailored for data analysis in this domain. However, current research predominantly revolves around visual recognition tasks, lacking comprehensive, large-scale image-text datasets that are aligned and suitable for training large VLMs, which poses significant challenges to effectively training such models for RS applications. In computer vision, recent research has demonstrated that fine-tuning large vision language models on small-scale, high-quality datasets can yield impressive performance in visual and language understanding. These results are comparable to state-of-the-art VLMs trained from scratch on massive amounts of data, such as GPT-4. Inspired by this captivating idea, in this work, we build a high-quality Remote Sensing Image Captioning dataset (RSICap) that facilitates the development of large VLMs in the RS field. Unlike previous RS datasets that either employ model-generated captions or short descriptions, RSICap comprises 2,585 human-annotated captions with rich and high-quality information. This dataset offers detailed descriptions for each image, encompassing scene descriptions (e.g., residential area, airport, or farmland) as well as object information (e.g., color, shape, quantity, absolute position, etc). To facilitate the evaluation of VLMs in the field of RS, we also provide a benchmark evaluation dataset called RSIEval. This dataset consists of human-annotated captions and visual question-answer pairs, allowing for a comprehensive assessment of VLMs in the context of RS.

Key facts. Cited by 204 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 93 references that are in the map. First posted on arXiv 28 Jul 2023.

Builds on

36 of 93 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Vision-Language Models in Remote Sensing: Current Progress and Future Trends9 May

Other references · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  10. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  12. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
23 more
  1. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  3. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  4. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  5. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  6. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  7. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  8. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  9. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  10. 2022Visual Spatial Reasoningcited by 288
  11. 2023MultiModal-GPT: A Vision and Language Model for Dialogue with Humanscited by 255
  12. 2020RSVQA: Visual Question Answering for Remote Sensing Datacited by 197
  13. 2017Exploring Models and Data for Remote Sensing Image Caption Generationcited by 195
  14. 2017Deep learning in remote sensing: a reviewcited by 128
  15. 2022Exploring a Fine-Grained Multiscale Method for Cross-Modal Remote Sensing Image Retrievalcited by 77
  16. 2022Remote Sensing Cross-Modal Text-Image Retrieval Based on Global and Local Informationcited by 40
  17. 2022From Easy to Hard: Learning Language-guided Curriculum for Visual Question Answering on Remote Sensing Datacited by 38
  18. 2021Change Detection Meets Visual Question Answeringcited by 29
  19. 2022Txt2Img-MHN: Remote Sensing Image Generation from Text Using Modern Hopfield Networkscited by 28
  20. 2023Meta Learning to Bridge Vision and Language Models for Multimodal Few-Shot Learningcited by 20
  21. 2020SD-RSIC: Summarization Driven Deep Remote Sensing Image Captioningcited by 19
  22. 2017Fine-Grained Object Recognition and Zero-Shot Learning in Remote Sensing Imagerycited by 17
  23. 2019Multi-modal gated recurrent units for image descriptioncited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 52; 2025: 81; 2026: 66.

204 papers.

Sort:

By area:

  1. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensingcited by 320
  2. 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domaincited by 181
  3. 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Modelcited by 133
  4. 2024VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understandingcited by 116
  5. 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Modelcited by 112
  6. 2024SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understandingcited by 101
  7. 2024VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysiscited by 84
  8. 2024EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialoguescited by 81
  9. 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMscited by 72
  10. 2023Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignmentcited by 71
  11. 2025Sparse Autoencoders Learn Monosemantic Features in Vision-Language Modelscited by 70
  12. 2024TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Datacited by 69
38 more
  1. 2024MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretrainingcited by 66
  2. 2025GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensingcited by 55
  3. 2024GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Taskscited by 50
  4. 2025XLRS-Bench: Could Your Multimodal LLMs Understand Extremely Large Ultra-High-Resolution Remote Sensing Imagery?cited by 50
  5. 2024RS-Agent: Automating Remote Sensing Tasks through Intelligent Agentcited by 46
  6. 2024A Survey on Evaluation of Multimodal Large Language Modelscited by 46
  7. 2025GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolutioncited by 41
  8. 2024Good at captioning, bad at counting: Benchmarking GPT-4V on Earth observation datacited by 39
  9. 2024Towards Vision-Language Geo-Foundation Model: A Surveycited by 36
  10. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 35
  11. 2025DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Responsecited by 35
  12. 2024Deep Learning for Cross-Domain Data Fusion in Urban Computing: Taxonomy, Advances, and Outlookcited by 33
  13. 2024On the Foundations of Earth and Climate Foundation Modelscited by 31
  14. 2024Large Language Models for Captioning and Retrieving Remote Sensing Imagescited by 30
  15. 2024RSUniVLM: A Unified Vision Language Model for Remote Sensing via Granularity-oriented Mixture of Expertscited by 29
  16. 2024UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarioscited by 28
  17. 2025Falcon: A Remote Sensing Vision-Language Foundation Model (Technical Report)cited by 28
  18. 2024Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectivescited by 27
  19. 2025A Survey of Multimodal Hallucination Evaluation and Detectioncited by 26
  20. 2024LHRS-Bot-Nova: Improved Multimodal Large Language Model for Remote Sensing Vision-Language Interpretationcited by 25
  21. 2024ChatEarthNet: A Global-Scale Image-Text Dataset Empowering Vision-Language Geo-Foundation Modelscited by 23
  22. 2024AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuningcited by 23
  23. 2024Continual LLaVA: Continual Instruction Tuning in Large Vision-Language Modelscited by 22
  24. 2024Remote Sensing SpatioTemporal Vision-Language Models: A Comprehensive Surveycited by 22
  25. 2024An LLM Agent for Automatic Geospatial Data Analysiscited by 21
  26. 2025Multi-Agent Geospatial Copilots for Remote Sensing Workflowscited by 20
  27. 2024CHOICE: Benchmarking the Remote Sensing Capabilities of Large Vision-Language Modelscited by 19
  28. 2025RingMo-Agent: A Unified Remote Sensing Foundation Model for Multi-Platform and Multi-Modal Reasoningcited by 19
  29. 2025Geo-R1: Improving Few-Shot Geospatial Referring Expression Understanding with Reinforcement Fine-Tuningcited by 19
  30. 2025A Survey on Remote Sensing Foundation Models: From Vision to Multimodalitycited by 18
  31. 2025Vision-Language Modeling Meets Remote Sensing: Models, Datasets and Perspectivescited by 18
  32. 2025EarthMind: Leveraging Cross-Sensor Data for Advanced Earth Observation Interpretation with a Unified Multimodal LLMcited by 18
  33. 2024Semantic-CC: Boosting Remote Sensing Image Change Captioning via Foundational Knowledge and Semantic Guidancecited by 17
  34. 2025RSVLM-QA: A Benchmark Dataset for Remote Sensing Vision Language Model-based Question Answeringcited by 17
  35. 2025SARLANG-1M: A Benchmark for Vision-Language Modeling in SAR Image Understandingcited by 16
  36. 2024RS-MoE: A Vision-Language Model with Mixture of Experts for Remote Sensing Image Captioning and Visual Question Answeringcited by 15
  37. 2025UrbanLLaVA: A Multi-modal Large Language Model for Urban Intelligence with Spatial Reasoning and Understandingcited by 15
  38. 2025GeoVLM-R1: Reinforcement Fine-Tuning for Improved Remote Sensing Reasoningcited by 15

50 most cited of 204 All 204 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing
  2. 2025A Vision Centric Remote Sensing Benchmark
  3. 2025GeoRSMLLM: A Multimodal Large Language Model for Vision-Language Tasks in Geoscience and Remote Sensing
  4. 2025Bring Remote Sensing Object Detect Into Nature Language Model: Using SFT Method
  5. 2025Co-Training Vision Language Models for Remote Sensing Multi-task Learning
  6. 2025Quality-Driven Curation of Remote Sensing Vision-Language Data via Learned Scoring Models
  7. 2025A Recipe for Improving Remote Sensing VLM Zero Shot Generalization
  8. 2024Pushing the Limits of Vision-Language Models in Remote Sensing without Human Annotations