Multi-Perspective Subimage CLIP with Keyword Guidance for Remote Sensing Image-Text Retrieval
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
Vision-Language Pre-training (VLP) models like CLIP have significantly advanced Remote Sensing Image-Text Retrieval (RSITR). However, existing methods predominantly rely on coarse-grained global alignment, which often overlooks the dense, multi-scale semantics inherent in overhead imagery. Moreover, adapting these heavy models via full fine-tuning incurs prohibitive computational costs and risks catastrophic forgetting. To address these challenges, we propose MPS-CLIP, a parameter-efficient framework designed to shift the retrieval paradigm from global matching to keyword-guided fine-grained alignment. Specifically, we leverage a Large Language Model (LLM) to extract core semantic keywords, guiding the Segment Anything Model (SamGeo) to generate semantically relevant sub-perspectives. To efficiently adapt the frozen backbone, we introduce a Gated Global Attention (G^2A) adapter, which captures global context and long-range dependencies with minimal overhead. Furthermore, a Multi-Perspective Representation (MPR) module aggregates these local cues into robust multi-perspective embeddings. The framework is optimized via a hybrid objective combining multi-perspective contrastive and weighted triplet losses, which dynamically selects maximum-response perspectives to suppress noise and enforce precise semantic matching. Extensive experiments on the RSICD and RSITMD benchmarks demonstrate that MPS-CLIP achieves state-of-the-art performance with 35.18% and 48.40% mean Recall (mR), respectively, significantly outperforming full fine-tuning baselines and recent competitive methods. Code is available at https://github.com/Lcrucial1f/MPS-CLIP.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 27 references that are in the map. First posted on arXiv 26 Jan 2026.
Builds on
13 of 27 references.
Closest ancestors · 1
Same direction, newest first
Other references · 12
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
- 2022Exploring a Fine-Grained Multiscale Method for Cross-Modal Remote Sensing Image Retrievalcited by 77
- 2023UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modelingcited by 40
- 2023DiffCR: A Fast Conditional Diffusion Framework for Cloud Removal from Optical Satellite Imagescited by 26
- 2023Parameter-Efficient Transfer Learning for Remote Sensing Image-Text Retrievalcited by 24
- 2023Direction-Oriented Visual-semantic Embedding Model for Remote Sensing Image-text Retrievalcited by 14
- 2023Subnetwork-to-go: Elastic Neural Network with Dynamic Training and Customizable Inferencecited by 9
- 2024Apollo: Band-sequence Modeling for High-Quality Audio Restorationcited by 8
- 2024Adapting Vision Foundation Models for Robust Cloud Segmentation in Remote Sensing Imagescited by 7
- 2025Dynamic Dictionary Learning for Remote Sensing Image Segmentationcited by 7
- 2024Efficient Remote Sensing with Harmonized Transfer Learning and Modality Alignmentcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Beyond Pixels: A Training-Free, Text-to-Text Framework for Remote Sensing Image Retrieval
- 2024PriorCLIP: Visual Prior Guided Vision-Language Model for Remote Sensing Image-Text Retrieval
- 2025DGTRSD & DGTRS-CLIP: A Dual-Granularity Remote Sensing Image-Text Dataset and Vision Language Foundation Model for Alignment
- 2024Multilingual Vision-Language Pre-training for the Remote Sensing Domain
- 2025Remote Sensing Large Vision-Language Model: Semantic-augmented Multi-level Alignment and Semantic-aware Expert Modeling
- 2026Fast-then-Fine: A Two-Stage Framework with Multi-Granular Representation for Cross-Modal Retrieval in Remote Sensing
- 2025A Vision Centric Remote Sensing Benchmark
- 2024LuoJiaHOG: A Hierarchy Oriented Geo-aware Image Caption Dataset for Remote Sensing Image-Text Retrival