mlsanityData to 7 Oct 2026

GRASP: Guided Region-Aware Sparse Prompting for Adapting MLLMs to Remote Sensing

arXiv 2601.17089 ↗First posted 23 Jan 2026cs.CVcited by 5

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

In recent years, Multimodal Large Language Models (MLLMs) have made significant progress in visual question answering tasks. However, directly applying existing fine-tuning methods to remote sensing (RS) images often leads to issues such as overfitting on background noise or neglecting target details. This is primarily due to the large-scale variations, sparse target distributions, and complex regional semantic features inherent in RS images. These challenges limit the effectiveness of MLLMs in RS tasks. To address these challenges, we propose a parameter-efficient fine-tuning (PEFT) strategy called Guided Region-Aware Sparse Prompting (GRASP). GRASP introduces spatially structured soft prompts associated with spatial blocks extracted from a frozen visual token grid. Through a question-guided sparse fusion mechanism, GRASP dynamically aggregates task-specific context into a compact global prompt, enabling the model to focus on relevant regions while filtering out background noise. Extensive experiments on multiple RSVQA benchmarks show that GRASP achieves competitive performance compared to existing fine-tuning and prompt-based methods while maintaining high parameter efficiency.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 41 of its 53 references that are in the map. First posted on arXiv 23 Jan 2026.

Builds on

41 of 53 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Remote Sensing Large Vision-Language Model: Semantic-augmented Multi-level Alignment and Semantic-aware Expert Modeling27 Jun
  2. 2025EarthGPT-X: A Spatial MLLM for Multi-level Multi-Source Remote Sensing Imagery Understanding with Visual Prompting17 Apr
  3. 2024Large Vision-Language Models for Remote Sensing Visual Question Answering16 Nov
  4. 2024From Pixels to Prose: Advancing Multi-Modal Language Models for Remote Sensing5 Nov
  5. 2024DDFAV: Remote Sensing Large Vision Language Models Dataset and Evaluation Benchmark5 Nov
  6. 2024EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensing18 Jul
  7. 2024Towards Vision-Language Geo-Foundation Model: A Survey13 Jun
  8. 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model4 Feb
  9. 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domain30 Jan
  10. 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model18 Jan
  11. 2023C-SAW: Self-Supervised Prompt Learning for Image Generalization in Remote Sensing27 Nov
  12. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensing24 Nov
2 more
  1. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensing19 Jun
  2. 2023Vision-Language Models in Remote Sensing: Current Progress and Future Trends9 May

Other references · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2015VQA: Visual Question Answeringcited by 2,489
  9. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  10. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  11. 2022Visual Prompt Tuningcited by 1,674
  12. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
15 more
  1. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  2. 2023A Survey on Multimodal Large Language Modelscited by 898
  3. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 740
  4. 2017DOTA: A Large-scale Dataset for Object Detection in Aerial Imagescited by 622
  5. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 277
  6. 2020RSVQA: Visual Question Answering for Remote Sensing Datacited by 197
  7. 2019Sparse Sequence-to-Sequence Modelscited by 167
  8. 2022Exploring a Fine-Grained Multiscale Method for Cross-Modal Remote Sensing Image Retrievalcited by 77
  9. 2024Foundation Models for Remote Sensing and Earth Observation: A Surveycited by 66
  10. 2025A Survey on Remote Sensing Foundation Models: From Vision to Multimodalitycited by 18
  11. 2022Multi-Modal Fusion Transformer for Visual Question Answering in Remote Sensingcited by 13
  12. 2023LiT-4-RSVQA: Lightweight Transformer-based Visual Question Answering in Remote Sensingcited by 10
  13. 2024Jointly RS Image Deblurring and Super-Resolution with Adjustable-Kernel and Multi-Domain Attentioncited by 10
  14. 2023MVP: Meta Visual Prompt Tuning for Few-Shot Remote Sensing Image Scene Classificationcited by 7
  15. 2023Visual Question Answering in Remote Sensing with Cross-Attention and Multimodal Information Bottleneckcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Transforming External Knowledge into Triplets for Enhanced Retrieval in RAG of LLMscited by 3
  2. 2026RCP: Representation Consistency Pruner for Mitigating Distribution Shift in Large Vision-Language Modelscited by 1
  3. 2026Immunizing 3D Gaussian Generative Models Against Unauthorized Fine-Tuning via Attribute-Space Trapscited by 1
  4. 2026DASH-KV: Accelerating Long-Context LLM Inference via Asymmetric KV Cache Hashingcited by 1
  5. 2026From Pixels to Hierarchical Sequences: Quadtree Mask Encoding for Vision-Language Binary Change Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Vision Centric Remote Sensing Benchmark
  2. 2026Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?
  3. 2026A Unified Framework for Efficient Remote Sensing Visual Question Answering: Adapting Dual, Hybrid, and Encoder-Decoder Architectures
  4. 2024RS-MoE: A Vision-Language Model with Mixture of Experts for Remote Sensing Image Captioning and Visual Question Answering
  5. 2026From Coordinates to Candidate Regions: Temporal Change Localization via Region Selection in Remote Sensing Multimodal LLMs
  6. 2025GeoRSMLLM: A Multimodal Large Language Model for Vision-Language Tasks in Geoscience and Remote Sensing
  7. 2026bi-modal textual prompt learning for vision-language models in remote sensing
  8. 2026More with Less: a Large Scale Remote Sensing VLM with a Simple Recipe