DescribeEarth: Describe Anything for Remote Sensing Images
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
Automated textual description of remote sensing images is crucial for unlocking their full potential in diverse applications, from environmental monitoring to urban planning and disaster management. However, existing studies in remote sensing image captioning primarily focus on the image level, lacking object-level fine-grained interpretation, which prevents the full utilization and transformation of the rich semantic and structural information contained in remote sensing images. To address this limitation, we propose Geo-DLC, a novel task of object-level fine-grained image captioning for remote sensing. To support this task, we construct DE-Dataset, a large-scale dataset contains 25 categories and 261,806 annotated instances with detailed descriptions of object attributes, relationships, and contexts. Furthermore, we introduce DE-Benchmark, a LLM-assisted question-answering based evaluation suite designed to systematically measure model capabilities on the Geo-DLC task. We also present DescribeEarth, a Multi-modal Large Language Model (MLLM) architecture explicitly designed for Geo-DLC, which integrates a scale-adaptive focal strategy and a domain-guided fusion module leveraging remote sensing vision-language model features to encode high-resolution details and remote sensing category priors while maintaining global context. Our DescribeEarth model consistently outperforms state-of-the-art general MLLMs on DE-Benchmark, demonstrating superior factual accuracy, descriptive richness, and grammatical soundness, particularly in capturing intrinsic object features and surrounding environmental attributes across simple, complex, and even out-of-distribution remote sensing scenarios. All data, code and weights are released at https://github.com/earth-insights/DescribeEarth.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 84 of its 110 references that are in the map. First posted on arXiv 30 Sep 2025.
Builds on
84 of 110 references.
Closest ancestors · 15
Same direction, newest first
- 2025EagleVision: Object-level Attribute Multimodal LLM for Remote Sensing30 Mar
- 2024UniRS: Unifying Multi-temporal Remote Sensing Tasks through Vision Language Models30 Dec
- 2024EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensing18 Jul
- 2024SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understanding14 Jun
- 2024VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysis29 Mar
- 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model4 Feb
- 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domain30 Jan
- 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model18 Jan
- 2023SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing20 Dec
- 2023Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignment12 Dec
- 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensing24 Nov
- 2023RSGPT: A Remote Sensing Vision Language Model and Benchmark28 Jul
3 more
- 2023RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing20 Jun
- 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensing19 Jun
- 2023APPLeNet: Visual Attention Parameterized Prompt Learning for Few-Shot Remote Sensing Image Generalization using CLIP12 Apr
Other references · 69
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2023Segment Anythingcited by 8,234
- 2023Visual Instruction Tuningcited by 7,869
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
38 more
- 2024A Survey on LLM-as-a-Judgecited by 1,423
- 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
- 2014Show and Tell: A Neural Image Caption Generatorcited by 879
- 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
- 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 797
- 2024Phi-4 Technical Reportcited by 711
- 2017DOTA: A Large-scale Dataset for Object Detection in Aerial Imagescited by 623
- 2021ClipCap: CLIP Prefix for Image Captioningcited by 559
- 2017Scene Graph Generation by Iterative Message Passingcited by 462
- 2016AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classificationcited by 347
- 2019Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmarkcited by 340
- 2018Graph R-CNN for Scene Graph Generationcited by 298
- 2019Attention on Attention for Image Captioningcited by 289
- 2021LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentationcited by 261
- 2016Image Captioning with Semantic Attentioncited by 260
- 2018Learning to Compose Dynamic Tree Structures for Visual Contextscited by 259
- 2018Auto-Encoding Scene Graphs for Image Captioningcited by 255
- 2023SpectralGPT: Spectral Remote Sensing Foundation Modelcited by 252
- 2018Exploring Visual Relationship for Image Captioningcited by 215
- 2020X-Linear Attention Networks for Image Captioningcited by 198
- 2017Exploring Models and Data for Remote Sensing Image Caption Generationcited by 197
- 2019iSAID: A Large-scale Dataset for Instance Segmentation in Aerial Imagescited by 197
- 2021UNetFormer: A UNet-like Transformer for Efficient Semantic Segmentation of Remote Sensing Urban Scene Imagerycited by 167
- 2024TinyLLaVA: A Framework of Small-scale Large Multimodal Modelscited by 164
- 2017Scene Graph Generation from Objects, Phrases and Region Captionscited by 154
- 2023SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagerycited by 151
- 2018xView: Objects in Context in Overhead Imagerycited by 146
- 2021FAIR1M: A Benchmark Dataset for Fine-grained Object Recognition in High-Resolution Remote Sensing Imagerycited by 143
- 2022RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Datacited by 123
- 2017Visual Translation Embedding Network for Visual Relation Detectioncited by 112
- 2019Scene Graph Generation with External Knowledge and Image Reconstructioncited by 102
- 2017Detecting Visual Relationships with Deep Relational Networkscited by 97
- 2025Describe Anything: Detailed Localized Image and Video Captioningcited by 84
- 2018Unsupervised Image Captioningcited by 79
- 2024Mission Critical -- Satellite Data is a Distinct Modality in Machine Learningcited by 77
- 2016Boosting Image Captioning with Attributescited by 76
- 2024TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Datacited by 70
- 2023RRSIS: Referring Remote Sensing Image Segmentationcited by 59
50 most cited of 69 All 69 (JSON)
Built on by
Citing papers per year: 2025: 2; 2026: 3.
5 papers.
- 2025EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewardscited by 27
- 2025Designing Domain-Specific Agents via Hierarchical Task Abstraction Mechanismcited by 16
- 2026Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Taskscited by 4
- 2026GeoAlignCLIP: Enhancing Fine-Grained Vision-Language Alignment in Remote Sensing via Multi-Granular Consistency Learningcited by 2
- 2026The Earth in One Gaze: Training-Free Active Focus for UHR Remote Sensing Understandingcited by 0
- 2026The Earth in One Gaze: Training-Free Active Focus for UHR Remote Sensing Understanding23 Sep 2026
- 2026Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks9 Jun 2026
- 2026GeoAlignCLIP: Enhancing Fine-Grained Vision-Language Alignment in Remote Sensing via Multi-Granular Consistency Learning10 Mar 2026
- 2025Designing Domain-Specific Agents via Hierarchical Task Abstraction Mechanism21 Nov 2025
- 2025EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewards20 Nov 2025
Similar, not cited
Close in wording, no citation either way
- 2025GeoRSMLLM: A Multimodal Large Language Model for Vision-Language Tasks in Geoscience and Remote Sensing
- 2026Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?
- 2025Beyond Pixels: A Training-Free, Text-to-Text Framework for Remote Sensing Image Retrieval
- 2024From Pixels to Prose: Advancing Multi-Modal Language Models for Remote Sensing
- 2026BigEarthNet.txt: A Large-Scale Multi-Sensor Image-Text Dataset and Benchmark for Earth Observation
- 2024LuoJiaHOG: A Hierarchy Oriented Geo-aware Image Caption Dataset for Remote Sensing Image-Text Retrival
- 2024VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding
- 2025Multilingual Training-Free Remote Sensing Image Captioning