mlsanityData to 7 Oct 2026

When Large Vision-Language Model Meets Large Remote Sensing Imagery: Coarse-to-Fine Text-Guided Token Pruning

arXiv 2503.07588 ↗First posted 10 Mar 2025cs.CV cs.AIcited by 35

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

Efficient vision-language understanding of large Remote Sensing Images (RSIs) is meaningful but challenging. Current Large Vision-Language Models (LVLMs) typically employ limited pre-defined grids to process images, leading to information loss when handling gigapixel RSIs. Conversely, using unlimited grids significantly increases computational costs. To preserve image details while reducing computational complexity, we propose a text-guided token pruning method with Dynamic Image Pyramid (DIP) integration. Our method introduces: (i) a Region Focus Module (RFM) that leverages text-aware region localization capability to identify critical vision tokens, and (ii) a coarse-to-fine image tile selection and vision token pruning strategy based on DIP, which is guided by RFM outputs and avoids directly processing the entire large imagery. Additionally, existing benchmarks for evaluating LVLMs' perception ability on large RSI suffer from limited question diversity and constrained image sizes. We construct a new benchmark named LRS-VQA, which contains 7,333 QA pairs across 8 categories, with image length up to 27,328 pixels. Our method outperforms existing high-resolution strategies on four datasets using the same data. Moreover, compared to existing token reduction methods, our approach demonstrates higher efficiency under high-resolution settings. Dataset and code are in https://github.com/VisionXLab/LRS-VQA.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 79 of its 95 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

79 of 95 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensing23 Jan
  2. 2025A Simple Aerial Detection Baseline of Multimodal Language Models16 Jan
  3. 2024REO-VLM: Transforming VLM to Meet Regression Challenges in Earth Observation21 Dec
  4. 2024EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialogues19 Dec
  5. 2024RSUniVLM: A Unified Vision Language Model for Remote Sensing via Granularity-oriented Mixture of Experts7 Dec
  6. 2024GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks28 Nov
  7. 2024EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensing18 Jul
  8. 2024SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understanding14 Jun
  9. 2024Towards Vision-Language Geo-Foundation Model: A Survey13 Jun
  10. 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model4 Feb
  11. 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domain30 Jan
  12. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensing24 Nov
1 more
  1. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensing19 Jun

Other references · 66

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023Segment Anythingcited by 8,158
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2024Qwen2.5 Technical Reportcited by 4,713
  9. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  10. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  11. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  12. 2022A ConvNet for the 2020scited by 3,545
38 more
  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  3. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  4. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  5. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  6. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  7. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Modelscited by 629
  8. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 539
  9. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 459
  10. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Modelscited by 394
  11. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inferencecited by 378
  12. 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 361
  13. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 356
  14. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 298
  15. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 284
  16. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reductioncited by 283
  17. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 267
  18. 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228
  19. 2021Object Detection in Aerial Images: A Large-Scale Benchmark and Challengescited by 202
  20. 2023SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagerycited by 202
  21. 2020RSVQA: Visual Question Answering for Remote Sensing Datacited by 197
  22. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  23. 2024LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Imagescited by 193
  24. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Documentcited by 173
  25. 2024TokenPacker: Efficient Visual Projector for Multimodal LLMcited by 170
  26. 2021SPViT: Enabling Faster Vision Transformers via Soft Token Pruningcited by 165
  27. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Tokencited by 164
  28. 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Modelscited by 151
  29. 2021FAIR1M: A Benchmark Dataset for Fine-grained Object Recognition in High-Resolution Remote Sensing Imagerycited by 143
  30. 2021Learned Token Pruning for Transformerscited by 138
  31. 2024Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Modelscited by 125
  32. 2022RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Datacited by 121
  33. 2024Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Modelscited by 112
  34. 2024LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via a Hybrid Architecturecited by 100
  35. 2024ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Explorationcited by 99
  36. 2024Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-trainingcited by 82
  37. 2023Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformerscited by 79
  38. 2024VoCo-LLaMA: Towards Vision Compression with Large Language Modelscited by 78

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 25.

35 papers.

Sort:

By area:

  1. 2025GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolutioncited by 41
  2. 2025RSVLM-QA: A Benchmark Dataset for Remote Sensing Vision Language Model-based Question Answeringcited by 17
  3. 2025GeoZero: Incentivizing Reasoning from Scratch on Geospatial Scenescited by 13
  4. 2025Look Where It Matters: Training-Free Ultra-HR Remote Sensing VQA via Adaptive Zoom Searchcited by 11
  5. 2025A Benchmark for Ultra-High-Resolution Remote Sensing MLLMscited by 11
  6. 2026GeoEyes: On-Demand Visual Focusing for Evidence-Grounded Understanding of Ultra-High-Resolution Remote Sensing Imagerycited by 9
  7. 2026GeoMMBench and GeoMMAgent: Toward Expert-Level Multimodal Intelligence in Geoscience and Remote Sensingcited by 6
  8. 2025VICoT-Agent: A Vision-Interleaved Chain-of-Thought Framework for Interpretable Multimodal Reasoning and Scalable Remote Sensing Analysiscited by 4
  9. 2026Text Before Vision: Staged Knowledge Injection Matters for Agentic RLVR in Ultra-High-Resolution Remote Sensing Understandingcited by 4
  10. 2025Co-Training Vision Language Models for Remote Sensing Multi-task Learningcited by 3
  11. 2025FUSE-RSVLM: Feature Fusion Vision-Language Model for Remote Sensingcited by 3
  12. 2026Seeing Clearly without Training: Mitigating Hallucinations in Multimodal LLMs for Remote Sensingcited by 3
23 more
  1. 2026OmniEarth: A Benchmark for Evaluating Vision-Language Models in Geospatial Taskscited by 3
  2. 2026Compression Tells Intelligence: Visual Coding, Visual Token Technology, and the Unificationcited by 2
  3. 2026OSMDA: OpenStreetMap-based Domain Adaptation for Remote Sensing VLMscited by 2
  4. 2026WeaveEarth: Structured Evidence Construction and Reasoning for Training-Free UHR Remote Sensing Understandingcited by 2
  5. 2025Remote Sensing Image Intelligent Interpretation with the Language-Centered Perspective: Principles, Methods and Challengescited by 1
  6. 2026UAV traffic scene understanding: A regulation embedded multi-modal network and a unified benchmarkcited by 1
  7. 2026NeSy-Route: A Neuro-Symbolic Benchmark for Constrained Route Planning in Remote Sensingcited by 1
  8. 2026Semantic-Geometric Dual Compression: Training-Free Visual Token Reduction for Ultra-High-Resolution Remote Sensing Understandingcited by 1
  9. 2026UHR-BAT: Budget-Aware Token Compression Vision-Language model for Ultra-High-Resolution Remote Sensingcited by 1
  10. 2026Agentic AI for Remote Sensing: Technical Challenges and Research Directionscited by 1
  11. 2026Beyond Zooming: Learning Multi-Tool Visual Reasoning for Ultra-High-Resolution Remote Sensingcited by 1
  12. 2025Cross-modal Context-aware Learning for Visual Prompt Guided Multimodal Image Understanding in Remote Sensingcited by 0
  13. 2026GeoVista: Visually Grounded Active Perception for Vision-Language Understanding of Ultra-High-Resolution Remote Sensing Imagescited by 0
  14. 2026Text-RSIR: A Text-Guided Framework for Efficient Remote Sensing Image Transmission and Reconstructioncited by 0
  15. 2026SkyNative: A Native Multimodal Architecture for Remote Sensing Vision-Language Understandingcited by 0
  16. 2026MonoIR-RS: Infrared Remote Sensing Vision-Language Learning with CLIP and VLM Adaptationcited by 0
  17. 2026More with Less: a Large Scale Remote Sensing VLM with a Simple Recipecited by 0
  18. 2026SA-GEM: Scale-Adaptive and Geospatial Evidence-Modulated Token Pruning for Efficient Remote Sensing Large Vision-Language Modelscited by 0
  19. 2026Exploiting answer-invariant redundancies in satellite imagery for efficient VLM inference on edgecited by 0
  20. 2026WeaveAgent: A Two-Stage Tool-Routing Agent for Ultra-High-Resolution Remote Sensing Imagerycited by 0
  21. 2026Can't Find Waldo: Evaluating VLMs' Sensitivity to Image Resolution and Detail Levelcited by 0
  22. 2026The Earth in One Gaze: Training-Free Active Focus for UHR Remote Sensing Understandingcited by 0
  23. 2026RS-OPSD: Reliable Privileged On-Policy-Self-Distillation for Ultra-High-Resolution Remote Sensing VQAcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model
  2. 2026Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?
  3. 2023RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing
  4. 2026RRS-10K: A Multitask Vision-Language Model Benchmark for Rare Remote Sensing Image Interpretation
  5. 2023RSGPT: A Remote Sensing Vision Language Model and Benchmark
  6. 2026GRASP: Guided Region-Aware Sparse Prompting for Adapting MLLMs to Remote Sensing
  7. 2025A Vision Centric Remote Sensing Benchmark
  8. 2026Continual Vision-Language Learning for Remote Sensing: Benchmarking and Analysis