mlsanityData to 14 Oct 2026

XLRS-Bench: Could Your Multimodal LLMs Understand Extremely Large Ultra-High-Resolution Remote Sensing Imagery?

arXiv 2503.23771 ↗First posted 31 Mar 2025cs.CVcited by 51

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

The astonishing breakthrough of multimodal large language models (MLLMs) has necessitated new benchmarks to quantitatively assess their capabilities, reveal their limitations, and indicate future research directions. However, this is challenging in the context of remote sensing (RS), since the imagery features ultra-high resolution that incorporates extremely complex semantic relationships. Existing benchmarks usually adopt notably smaller image sizes than real-world RS scenarios, suffer from limited annotation quality, and consider insufficient dimensions of evaluation. To address these issues, we present XLRS-Bench: a comprehensive benchmark for evaluating the perception and reasoning capabilities of MLLMs in ultra-high-resolution RS scenarios. XLRS-Bench boasts the largest average image size (8500×8500) observed thus far, with all evaluation samples meticulously annotated manually, assisted by a novel semi-automatic captioner on ultra-high-resolution RS images. On top of the XLRS-Bench, 16 sub-tasks are defined to evaluate MLLMs' 10 kinds of perceptual capabilities and 6 kinds of reasoning capabilities, with a primary emphasis on advanced cognitive processes that facilitate real-world decision-making and the capture of spatiotemporal changes. The results of both general and RS-focused MLLMs on XLRS-Bench indicate that further efforts are needed for real-world RS applications. We have open-sourced XLRS-Bench to support further research in developing more powerful MLLMs for remote sensing.

Key facts. Cited by 51 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 53 of its 94 references that are in the map. First posted on arXiv 31 Mar 2025.

Builds on

53 of 94 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding18 Jun
  2. 2024SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understanding14 Jun
  3. 2024Efficient Prompt Tuning of Large Vision-Language Model for Fine-Grained Ship Classification13 Mar
  4. 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model4 Feb
  5. 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domain30 Jan
  6. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensing24 Nov
  7. 2023RSGPT: A Remote Sensing Vision Language Model and Benchmark28 Jul

Other references · 46

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  7. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  8. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  9. 2019Towards VQA Models That Can Readcited by 1,718
  10. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  11. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  12. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
34 more
  1. 2018Datasheets for Datasetscited by 1,386
  2. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  3. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
  4. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  5. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  6. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  7. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  8. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  9. 2017DOTA: A Large-scale Dataset for Object Detection in Aerial Imagescited by 623
  10. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
  11. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
  12. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
  13. 2018nocaps: novel object captioning at scalecited by 444
  14. 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 365
  15. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 357
  16. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
  17. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  18. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 292
  19. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimizationcited by 239
  20. 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 217
  21. 2020RSVQA: Visual Question Answering for Remote Sensing Datacited by 203
  22. 2017Exploring Models and Data for Remote Sensing Image Caption Generationcited by 197
  23. 2024LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Imagescited by 195
  24. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  25. 2024MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGIcited by 193
  26. 2022RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Datacited by 123
  27. 2023VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Usecited by 102
  28. 2024MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answeringcited by 93
  29. 2023Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentationcited by 76
  30. 2024Beyond LLaVA-HD: Diving into High-Resolution Large Multimodal Modelscited by 76
  31. 2024PANGAEA: A Global and Inclusive Benchmark for Geospatial Foundation Modelscited by 68
  32. 2023TouchStone: Evaluating Vision-Language Models by Language Modelscited by 55
  33. 2020Semi-Supervised Semantic Segmentation in Earth Observation: The MiniFrance Suite, Dataset Analysis and Multi-task Network Studycited by 28
  34. 2025RoMA: Scaling up Mamba-based Foundation Models for Remote Sensingcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 17; 2026: 34.

51 papers.

Sort:

By area:

  1. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 90
  2. 2025Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodologycited by 69
  3. 2025Intern-S1: A Scientific Multimodal Foundation Modelcited by 53
  4. 2025GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolutioncited by 44
  5. 2025SegEarth-R1: Geospatial Pixel Reasoning via Large Language Modelcited by 42
  6. 2025DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Responsecited by 36
  7. 2026Intern-S1-Pro: Scientific Multimodal Foundation Model at Trillion Scalecited by 29
  8. 2025RingMo-Agent: A Unified Remote Sensing Foundation Model for Multi-Platform and Multi-Modal Reasoningcited by 20
  9. 2025EarthMind: Leveraging Cross-Sensor Data for Advanced Earth Observation Interpretation with a Unified Multimodal LLMcited by 18
  10. 2025OmniEarth-Bench: Towards Holistic Evaluation of Earth's Six Spheres and Cross-Spheres Interactions with Multimodal Observational Earth Datacited by 14
  11. 2025RSCC: A Large-Scale Remote Sensing Change Caption Dataset for Disaster Eventscited by 14
  12. 2025GeoZero: Incentivizing Reasoning from Scratch on Geospatial Scenescited by 14
38 more
  1. 2025Look Where It Matters: Training-Free Ultra-HR Remote Sensing VQA via Adaptive Zoom Searchcited by 11
  2. 2025A Benchmark for Ultra-High-Resolution Remote Sensing MLLMscited by 11
  3. 2025UniGeoSeg: Towards Unified Open-World Segmentation for Geospatial Scenescited by 10
  4. 2025MM-UAVBench: How Well Do Multimodal Large Language Models See, Think, and Plan in Low-Altitude UAV Scenarios?cited by 10
  5. 2026GeoEyes: On-Demand Visual Focusing for Evidence-Grounded Understanding of Ultra-High-Resolution Remote Sensing Imagerycited by 9
  6. 2026OpenEarth-Agent: From Tool Calling to Tool Creation for Open-Environment Earth Observationcited by 9
  7. 2026Innovator-VL: A Multimodal Large Language Model for Scientific Discoverycited by 8
  8. 2026SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMscited by 7
  9. 2026TerraScope: Pixel-Grounded Visual Reasoning for Earth Observationcited by 7
  10. 2026VLRS-Bench: A Vision-Language Reasoning Benchmark for Remote Sensingcited by 6
  11. 2026GeoMMBench and GeoMMAgent: Toward Expert-Level Multimodal Intelligence in Geoscience and Remote Sensingcited by 6
  12. 2026Text Before Vision: Staged Knowledge Injection Matters for Agentic RLVR in Ultra-High-Resolution Remote Sensing Understandingcited by 4
  13. 2026Seeing Clearly without Training: Mitigating Hallucinations in Multimodal LLMs for Remote Sensingcited by 4
  14. 2026Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Taskscited by 4
  15. 2026Intern-S2-Preview: Scientific Agentic Foundation Modelcited by 4
  16. 2025MPCC: A Novel Benchmark for Multimodal Planning with Complex Constraints in Multimodal Large Language Modelscited by 2
  17. 2026NeSy-Route: A Neuro-Symbolic Benchmark for Constrained Route Planning in Remote Sensingcited by 2
  18. 2026RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioningcited by 2
  19. 2025Self-Supervised Enhancement of Forward-Looking Sonar Images: Bridging Cross-Modal Degradation Gaps through Feature Space Transformation and Multi-Frame Fusioncited by 1
  20. 2026Semantic-Geometric Dual Compression: Training-Free Visual Token Reduction for Ultra-High-Resolution Remote Sensing Understandingcited by 1
  21. 2026UHR-BAT: Budget-Aware Token Compression Vision-Language model for Ultra-High-Resolution Remote Sensingcited by 1
  22. 2026RSRCC: A Remote Sensing Regional Change Comprehension Benchmark Constructed via Retrieval-Augmented Best-of-N Rankingcited by 1
  23. 2026Beyond GSD-as-Token: Continuous Scale Conditioning for Remote Sensing VLMscited by 1
  24. 2026UHR-Micro: Diagnosing and Mitigating the Resolution Illusion in Earth Observation VLMscited by 1
  25. 2026Beyond Zooming: Learning Multi-Tool Visual Reasoning for Ultra-High-Resolution Remote Sensingcited by 1
  26. 2026Physics-Guided VLM Priors for All-Cloud Removalcited by 0
  27. 2026Sentinel2Cap: A Human-Annotated Benchmark Dataset for Multimodal Remote Sensing Image Captioningcited by 0
  28. 2026Beyond Appearance: Can Multimodal Large Language Models Exploit Vertical Structure for Remote Sensing Natural Scene Understanding?cited by 0
  29. 2026More with Less: a Large Scale Remote Sensing VLM with a Simple Recipecited by 0
  30. 2026Obshazard-bench: Benchmarking Multimodal Foundation Models for Real-Time Disaster Intelligence from Raw Earth Observation Streamscited by 0
  31. 2026EarthVerse: Benchmarking Scientific Agents Across Dynamic Earth Systems and Natural Hazardscited by 0
  32. 2026LogiScope-VQA: Benchmarking Vision-Language Models for Logistics Hazard Identification in Industrial Scenarioscited by 0
  33. 2026Earth-Agent-Pro: Towards Real-World Full-Chain Earth Observation with Agentscited by 0
  34. 2026GeoNLI - A Natural Language Interpreter for Satellite Imagerycited by 0
  35. 2026The Earth in One Gaze: Training-Free Active Focus for UHR Remote Sensing Understandingcited by 0
  36. 2026USAI-Quant: A Quantitative Reasoning Benchmark for Vision-Language Models in Built Environmentscited by 0
  37. 2026RS-OPSD: Reliable Privileged On-Policy-Self-Distillation for Ultra-High-Resolution Remote Sensing VQAcited by 0
  38. 2026Aperture: Training-Free Multiscale Concept Bottlenecks for Remote Sensingcited by 0

50 most cited of 51 All 51 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?
  2. 2025A Vision Centric Remote Sensing Benchmark
  3. 2024LHRS-Bot-Nova: Improved Multimodal Large Language Model for Remote Sensing Vision-Language Interpretation
  4. 2025Foundation Models for Remote Sensing: An Analysis of MLLMs for Object Localization
  5. 2024From Pixels to Prose: Advancing Multi-Modal Language Models for Remote Sensing
  6. 2025Remote Sensing Large Vision-Language Model: Semantic-augmented Multi-level Alignment and Semantic-aware Expert Modeling
  7. 2025TinyRS-R1: Compact Multimodal Language Model for Remote Sensing
  8. 2024DDFAV: Remote Sensing Large Vision Language Models Dataset and Evaluation Benchmark