Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
The rapid development of multimodal large language models (MLLMs) has introduced a flexible paradigm for remote sensing image scene understanding (RSISU), enabling natural-language interaction with remote sensing imagery. However, a systematic understanding of the capability boundaries, cross-task generalization, and task-specific limitations of existing remote sensing MLLMs (RS-MLLMs) is still lacking. This paper presents a systematic survey and diagnostic evaluation of MLLMs for RSISU. We review the technical evolution of RS-MLLMs, focusing on model design, multimodal learning, training data, and downstream capabilities. We further compare RS-MLLMs with general-purpose computer vision MLLMs (CV-MLLMs) across diverse RSISU tasks and benchmarks. RS-MLLMs remain competitive in domain-specific settings, particularly remote sensing visual grounding and high-resolution visual question answering. More notably, general-purpose CV-MLLMs can match or even outperform these specialized models on several RSISU tasks without remote sensing-specific fine-tuning. These findings demonstrate the strong transferability of general-purpose CV-MLLMs and show that current RS-MLLMs do not consistently outperform them across diverse RSISU tasks. Current MLLMs also face limitations in spatial and relational reasoning, fine-grained visual understanding, instruction diversity, and generalization across heterogeneous task formats. Based on these findings, we outline future directions toward reliable evaluation, multimodal and high-resolution reasoning, efficient deployment, and tool-augmented remote sensing agents. This survey provides a systematic reference for developing robust, generalizable, and practical MLLMs for RSISU.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 69 of its 87 references that are in the map. First posted on arXiv 22 Jul 2026.
Builds on
69 of 87 references.
Closest ancestors · 25
Same direction, newest first
- 2026Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks9 Jun
- 2026RS-Claw: Progressive Active Tool Exploration via Hierarchical Skill Trees for Remote Sensing Agents13 May
- 2026Beyond GSD-as-Token: Continuous Scale Conditioning for Remote Sensing VLMs8 May
- 2026RemoteShield: Enable Robust Multimodal Large Language Models for Earth Observation19 Apr
- 2025SAR-TEXT: A Large-Scale SAR Image-Text Dataset Built with SAR-Narrator and A Progressive Learning Strategy for Downstream Tasks24 Jul
- 2025Enhancing Remote Sensing Vision-Language Models Through MLLM and LLM-Based High-Quality Image-Text Dataset Generation22 Jul
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning1 Jul
- 2025GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution27 May
- 2025EarthGPT-X: A Spatial MLLM for Multi-level Multi-Source Remote Sensing Imagery Understanding with Visual Prompting17 Apr
- 2025Falcon: A Remote Sensing Vision-Language Foundation Model (Technical Report)14 Mar
- 2024RS-GPT4V: A Unified Multimodal Instruction-Following Dataset for Remote Sensing Image Understanding18 Jun
- 2024VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding18 Jun
13 more
- 2024SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understanding14 Jun
- 2024Towards Vision-Language Geo-Foundation Model: A Survey13 Jun
- 2024RS-Agent: Automating Remote Sensing Tasks through Intelligent Agent11 Jun
- 2024VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysis29 Mar
- 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model4 Feb
- 2024Good at captioning, bad at counting: Benchmarking GPT-4V on Earth observation data31 Jan
- 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domain30 Jan
- 2023SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing20 Dec
- 2023Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignment12 Dec
- 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensing24 Nov
- 2023RSGPT: A Remote Sensing Vision Language Model and Benchmark28 Jul
- 2023RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing20 Jun
- 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensing19 Jun
Other references · 44
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
- 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
- 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 408
- 2016AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classificationcited by 347
- 2022Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learningcited by 308
32 more
- 2023SpectralGPT: Spectral Remote Sensing Foundation Modelcited by 252
- 2023SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagerycited by 205
- 2020RSVQA: Visual Question Answering for Remote Sensing Datacited by 203
- 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 193
- 2020Geography-Aware Self-Supervised Learningcited by 190
- 2024TinyLLaVA: A Framework of Small-scale Large Multimodal Modelscited by 164
- 2022Advancing Plain Vision Transformer Towards Remote Sensing Foundation Modelcited by 149
- 2025Magma: A Foundation Model for Multimodal AI Agentscited by 139
- 2022An Empirical Study of Remote Sensing Pretrainingcited by 105
- 2023On the Opportunities and Challenges of Foundation Models for Geospatial Artificial Intelligencecited by 88
- 2023NExT-Chat: An LMM for Chat, Detection and Segmentationcited by 80
- 2023Predicting Gradient is Better: Exploring Self-Supervised Learning for SAR ATR with a Joint-Embedding Predictive Architecturecited by 46
- 2025ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Taskscited by 46
- 2025Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agentscited by 46
- 2024Large Language Models for Captioning and Retrieving Remote Sensing Imagescited by 30
- 2024GeoLLM-Engine: A Realistic Environment for Building Geospatial Copilotscited by 30
- 2019Relation Network for Multi-label Aerial Image Classificationcited by 26
- 2025SkySense V2: A Unified Foundation Model for Multi-modal Remote Sensingcited by 26
- 2024SARATR-X: Toward Building A Foundation Model for SAR Target Recognitioncited by 25
- 2025TerraFM: A Scalable Foundation Model for Unified Multisensor Earth Observationcited by 25
- 2025Multi-Agent Geospatial Copilots for Remote Sensing Workflowscited by 22
- 2025ZoomEarth: Active Perception for Ultra-High-Resolution Geospatial Vision-Language Taskscited by 21
- 2025RingMo-Agent: A Unified Remote Sensing Foundation Model for Multi-Platform and Multi-Modal Reasoningcited by 20
- 2025Geo-R1: Improving Few-Shot Geospatial Referring Expression Understanding with Reinforcement Fine-Tuningcited by 20
- 2024Evaluating Tool-Augmented Agents in Remote Sensing Platformscited by 17
- 2026Graph is a Substrate Across Data Modalitiescited by 14
- 2023The Potential of Visual ChatGPT For Remote Sensingcited by 10
- 2024ShapefileGPT: A Multi-Agent Large Language Model Framework for Automated Shapefile Processingcited by 9
- 2025SARMAE: Masked Autoencoder for SAR Representation Learningcited by 8
- 2026Same Signal, Opposite Meaning: Direction-Informed Adaptive Learning for LLM Agentscited by 8
- 2023Structured Generative Models for Scene Understandingcited by 3
- 2025Survey of Multimodal Geospatial Foundation Models: Techniques, Applications, and Challengescited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025A Vision Centric Remote Sensing Benchmark
- 2024From Pixels to Prose: Advancing Multi-Modal Language Models for Remote Sensing
- 2025A Benchmark for Ultra-High-Resolution Remote Sensing MLLMs
- 2025GeoRSMLLM: A Multimodal Large Language Model for Vision-Language Tasks in Geoscience and Remote Sensing
- 2024RSUniVLM: A Unified Vision Language Model for Remote Sensing via Granularity-oriented Mixture of Experts
- 2025Vision-Language Modeling Meets Remote Sensing: Models, Datasets and Perspectives
- 2025Co-Training Vision Language Models for Remote Sensing Multi-task Learning
- 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model