mlsanityData to 7 Oct 2026

LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model

arXiv 2402.02544 ↗First posted 4 Feb 2024cs.CV cs.AI cs.LGcited by 133

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

The revolutionary capabilities of large language models (LLMs) have paved the way for multimodal large language models (MLLMs) and fostered diverse applications across various specialized domains. In the remote sensing (RS) field, however, the diverse geographical landscapes and varied objects in RS imagery are not adequately considered in recent MLLM endeavors. To bridge this gap, we construct a large-scale RS image-text dataset, LHRS-Align, and an informative RS-specific instruction dataset, LHRS-Instruct, leveraging the extensive volunteered geographic information (VGI) and globally available RS images. Building on this foundation, we introduce LHRS-Bot, an MLLM tailored for RS image understanding through a novel multi-level vision-language alignment strategy and a curriculum learning method. Additionally, we introduce LHRS-Bench, a benchmark for thoroughly evaluating MLLMs' abilities in RS image understanding. Comprehensive experiments demonstrate that LHRS-Bot exhibits a profound understanding of RS images and the ability to perform nuanced reasoning within the RS domain.

Key facts. Cited by 133 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 71 of its 86 references that are in the map. First posted on arXiv 4 Feb 2024.

Builds on

71 of 86 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model18 Jan
  2. 2023SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing20 Dec
  3. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensing24 Nov
  4. 2023RSGPT: A Remote Sensing Vision Language Model and Benchmark28 Jul
  5. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensing19 Jun
  6. 2023Vision-Language Models in Remote Sensing: Current Progress and Future Trends9 May
  7. 2023PaLM-E: An Embodied Multimodal Language Model6 Mar

Other references · 64

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  9. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  12. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
38 more
  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  2. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  3. 2023Qwen Technical Reportcited by 3,273
  4. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  7. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  8. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  9. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  10. 2024Mixtral of Expertscited by 1,785
  11. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  12. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  13. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  14. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  15. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  16. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  17. 2022Token Merging: Your ViT But Fastercited by 995
  18. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  19. 2023BloombergGPT: A Large Language Model for Financecited by 923
  20. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
  21. 2017DOTA: A Large-scale Dataset for Object Detection in Aerial Imagescited by 622
  22. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  23. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 382
  24. 2016AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classificationcited by 343
  25. 2017Functional Map of the Worldcited by 338
  26. 2022Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learningcited by 302
  27. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  28. 2023Demystifying CLIP Datacited by 265
  29. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  30. 2023FinGPT: Open-Source Financial Large Language Modelscited by 244
  31. 2020RSVQA: Visual Question Answering for Remote Sensing Datacited by 197
  32. 2017Exploring Models and Data for Remote Sensing Image Caption Generationcited by 195
  33. 2022Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Modelscited by 172
  34. 2023Dolphins: Multimodal Language Model for Drivingcited by 124
  35. 2022RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Datacited by 121
  36. 2023VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Usecited by 100
  37. 2023SEED-Bench-2: Benchmarking Multimodal Large Language Modelscited by 96
  38. 2023XuanYuan 2.0: A Large Chinese Financial Chat Model with Hundreds of Billions Parameterscited by 92

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 28; 2025: 63; 2026: 42.

133 papers.

Sort:

By area:

  1. 2024SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understandingcited by 101
  2. 2024VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysiscited by 84
  3. 2024EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialoguescited by 81
  4. 2024TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Datacited by 69
  5. 2025ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Modelscited by 56
  6. 2025GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensingcited by 55
  7. 2024GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Taskscited by 50
  8. 2025XLRS-Bench: Could Your Multimodal LLMs Understand Extremely Large Ultra-High-Resolution Remote Sensing Imagery?cited by 50
  9. 2024RS-Agent: Automating Remote Sensing Tasks through Intelligent Agentcited by 46
  10. 2024GeoGround: A Unified Large Vision-Language Model for Remote Sensing Visual Groundingcited by 46
  11. 2025GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolutioncited by 41
  12. 2025RemoteSAM: Towards Segment Anything for Earth Observationcited by 38
38 more
  1. 2024Towards Vision-Language Geo-Foundation Model: A Surveycited by 36
  2. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 35
  3. 2025When Large Vision-Language Model Meets Large Remote Sensing Imagery: Coarse-to-Fine Text-Guided Token Pruningcited by 35
  4. 2024On the Foundations of Earth and Climate Foundation Modelscited by 31
  5. 2024EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensingcited by 29
  6. 2024RSUniVLM: A Unified Vision Language Model for Remote Sensing via Granularity-oriented Mixture of Expertscited by 29
  7. 2024UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarioscited by 28
  8. 2025Falcon: A Remote Sensing Vision-Language Foundation Model (Technical Report)cited by 28
  9. 2024GeoLLM-Engine: A Realistic Environment for Building Geospatial Copilotscited by 27
  10. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Modelscited by 27
  11. 2024SARATR-X: Toward Building A Foundation Model for SAR Target Recognitioncited by 25
  12. 2024LHRS-Bot-Nova: Improved Multimodal Large Language Model for Remote Sensing Vision-Language Interpretationcited by 25
  13. 2024Remote Sensing SpatioTemporal Vision-Language Models: A Comprehensive Surveycited by 22
  14. 2024CHOICE: Benchmarking the Remote Sensing Capabilities of Large Vision-Language Modelscited by 19
  15. 2025RingMo-Agent: A Unified Remote Sensing Foundation Model for Multi-Platform and Multi-Modal Reasoningcited by 19
  16. 2025A Survey on Remote Sensing Foundation Models: From Vision to Multimodalitycited by 18
  17. 2025Vision-Language Modeling Meets Remote Sensing: Models, Datasets and Perspectivescited by 18
  18. 2025EarthMind: Leveraging Cross-Sensor Data for Advanced Earth Observation Interpretation with a Unified Multimodal LLMcited by 18
  19. 2025DynamicVL: Benchmarking Multimodal Large Language Models for Dynamic City Understandingcited by 17
  20. 2025UrbanLLaVA: A Multi-modal Large Language Model for Urban Intelligence with Spatial Reasoning and Understandingcited by 15
  21. 2025GeoVLM-R1: Reinforcement Fine-Tuning for Improved Remote Sensing Reasoningcited by 15
  22. 2025Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methodscited by 15
  23. 2025OmniEarth-Bench: Towards Holistic Evaluation of Earth's Six Spheres and Cross-Spheres Interactions with Multimodal Observational Earth Datacited by 14
  24. 2025REOBench: Benchmarking Robustness of Earth Observation Foundation Modelscited by 13
  25. 2026Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodalitycited by 12
  26. 2025EarthGPT-X: A Spatial MLLM for Multi-level Multi-Source Remote Sensing Imagery Understanding with Visual Promptingcited by 11
  27. 2025Look Where It Matters: Training-Free Ultra-HR Remote Sensing VQA via Adaptive Zoom Searchcited by 11
  28. 2025A Benchmark for Ultra-High-Resolution Remote Sensing MLLMscited by 11
  29. 2025GeoPix: Multi-Modal Large Language Model for Pixel-level Image Understanding in Remote Sensingcited by 10
  30. 2025Quality-Driven Curation of Remote Sensing Vision-Language Data via Learned Scoring Modelscited by 10
  31. 2025UniGeoSeg: Towards Unified Open-World Segmentation for Geospatial Scenescited by 10
  32. 2024RS-GPT4V: A Unified Multimodal Instruction-Following Dataset for Remote Sensing Image Understandingcited by 9
  33. 2024REO-VLM: Transforming VLM to Meet Regression Challenges in Earth Observationcited by 9
  34. 2024Enhancing Remote Sensing Vision-Language Models for Zero-Shot Scene Classificationcited by 8
  35. 2024UniRS: Unifying Multi-temporal Remote Sensing Tasks through Vision Language Modelscited by 8
  36. 2025EagleVision: Object-level Attribute Multimodal LLM for Remote Sensingcited by 8
  37. 2025InstructSAM: A Training-Free Framework for Instruction-Oriented Remote Sensing Object Recognitioncited by 8
  38. 2024Multilingual Vision-Language Pre-training for the Remote Sensing Domaincited by 7

50 most cited of 133 All 133 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024RSTeller: Scaling Up Visual Language Modeling in Remote Sensing with Rich Linguistic Semantics from Openly Available Data and Large Language Models
  2. 2023RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing
  3. 2025Enhancing Remote Sensing Vision-Language Models Through MLLM and LLM-Based High-Quality Image-Text Dataset Generation
  4. 2025A Vision Centric Remote Sensing Benchmark
  5. 2026SkyNative: A Native Multimodal Architecture for Remote Sensing Vision-Language Understanding
  6. 2025A Benchmark for Multi-Lingual Vision-Language Learning in Remote Sensing Image Captioning
  7. 2026RemoteShield: Enable Robust Multimodal Large Language Models for Earth Observation
  8. 2026Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks