mlsanityData to 7 Oct 2026

Advancements in Visual Language Models for Remote Sensing: Datasets, Capabilities, and Enhancement Techniques

arXiv 2410.17283 ↗First posted 15 Oct 2024cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Recently, the remarkable success of ChatGPT has sparked a renewed wave of interest in artificial intelligence (AI), and the advancements in visual language models (VLMs) have pushed this enthusiasm to new heights. Differring from previous AI approaches that generally formulated different tasks as discriminative models, VLMs frame tasks as generative models and align language with visual information, enabling the handling of more challenging problems. The remote sensing (RS) field, a highly practical domain, has also embraced this new trend and introduced several VLM-based RS methods that have demonstrated promising performance and enormous potential. In this paper, we first review the fundamental theories related to VLM, then summarize the datasets constructed for VLMs in remote sensing and the various tasks they addressed. Finally, we categorize the improvement methods into three main parts according to the core components of VLMs and provide a detailed introduction and comparison of these methods. A project associated with this review has been created at https://github.com/taolijie11111/VLMs-in-RS-review.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 65 of its 98 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

65 of 98 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding18 Jun
  2. 2024SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understanding14 Jun
  3. 2024GeoReasoner: Geo-localization with Reasoning in Street Views using a Large Vision-Language Model3 Jun
  4. 2024VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysis29 Mar
  5. 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model4 Feb
  6. 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domain30 Jan
  7. 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model18 Jan
  8. 2023SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing20 Dec
  9. 2023Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignment12 Dec
  10. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensing24 Nov
  11. 2023RSGPT: A Remote Sensing Vision Language Model and Benchmark28 Jul
  12. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensing19 Jun
2 more
  1. 2023SATIN: A Multi-Task Metadataset for Classifying Satellite Imagery using Vision-Language Models23 Apr
  2. 2023APPLeNet: Visual Attention Parameterized Prompt Learning for Few-Shot Remote Sensing Image Generalization using CLIP12 Apr

Other references · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  9. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  10. 2023Visual Instruction Tuningcited by 7,792
  11. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  12. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
38 more
  1. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  2. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  7. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  8. 2015VQA: Visual Question Answeringcited by 2,489
  9. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  10. 2018Self-Attention with Relative Position Representationscited by 1,112
  11. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
  12. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  13. 2017DOTA: A Large-scale Dataset for Object Detection in Aerial Imagescited by 622
  14. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 617
  15. 2016AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classificationcited by 343
  16. 2019Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmarkcited by 339
  17. 2023SpectralGPT: Spectral Remote Sensing Foundation Modelcited by 250
  18. 2023SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagerycited by 202
  19. 2020RSVQA: Visual Question Answering for Remote Sensing Datacited by 197
  20. 2017Exploring Models and Data for Remote Sensing Image Caption Generationcited by 195
  21. 2019iSAID: A Large-scale Dataset for Instance Segmentation in Aerial Imagescited by 195
  22. 2023GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localizationcited by 185
  23. 2022SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understandingcited by 164
  24. 2021FAIR1M: A Benchmark Dataset for Fine-grained Object Recognition in High-Resolution Remote Sensing Imagerycited by 143
  25. 2022RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Datacited by 121
  26. 2023Towards Geospatial Foundation Models via Continual Pretrainingcited by 109
  27. 2023Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentationcited by 76
  28. 2023RRSIS: Referring Remote Sensing Image Segmentationcited by 59
  29. 2023EarthVQA: Towards Queryable Earth via Relational Reasoning-Based Remote Sensing Visual Question Answeringcited by 51
  30. 2024Large Language Models for Captioning and Retrieving Remote Sensing Imagescited by 30
  31. 2023S-CLIP: Semi-supervised Vision-Language Learning using Few Specialist Captionscited by 29
  32. 2022Txt2Img-MHN: Remote Sensing Image Generation from Text Using Modern Hopfield Networkscited by 28
  33. 2023CPSeg: Finer-grained Image Semantic Segmentation via Chain-of-Thought Language Promptingcited by 28
  34. 2024ChangeChat: An Interactive Model for Remote Sensing Change Analysis via Multimodal Instruction Tuningcited by 23
  35. 2023MGeo: Multi-Modal Geographic Pre-Training Methodcited by 12
  36. 2024Panoptic Perception: A Novel Task and Fine-grained Dataset for Universal Remote Sensing Image Interpretationcited by 8
  37. 2024TG-LLaVA: Text Guided LLaVA via Learnable Latent Embeddingscited by 8
  38. 2024ProGEO: Generating Prompts through Image-Text Contrastive Learning for Visual Geo-localizationcited by 6

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Remote Sensing SpatioTemporal Vision-Language Models: A Comprehensive Surveycited by 22

Similar, not cited

Close in wording, no citation either way

  1. 2025Vision-Language Modeling Meets Remote Sensing: Models, Datasets and Perspectives
  2. 2023Vision-Language Models in Remote Sensing: Current Progress and Future Trends
  3. 2024From Pixels to Prose: Advancing Multi-Modal Language Models for Remote Sensing
  4. 2026Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?
  5. 2025A Recipe for Improving Remote Sensing VLM Zero Shot Generalization
  6. 2025Bring Remote Sensing Object Detect Into Nature Language Model: Using SFT Method
  7. 2025Efficient Few-Shot Learning in Remote Sensing: Fusing Vision and Vision-Language Models
  8. 2025FUSE-RSVLM: Feature Fusion Vision-Language Model for Remote Sensing