# RSVG-ZeroOV: Exploring a Training-Free Framework for Zero-Shot Open-Vocabulary Visual Grounding in Remote Sensing Images

arXiv: https://arxiv.org/abs/2509.18711 · HTML: https://mlsanity.com/paper/2509.18711/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 23 Sep 2025.

## Builds on (most cited first, 14 of 14)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [RemoteCLIP: A Vision Language Foundation Model for Remote Sensing](https://mlsanity.com/paper/2306.11029/) (2023)
- [Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark](https://mlsanity.com/paper/1909.00133/) (2019)
- [GeoChat: Grounded Large Vision-Language Model for Remote Sensing](https://mlsanity.com/paper/2311.15826/) (2023)
- [RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Model](https://mlsanity.com/paper/2306.16269/) (2023)
- [Locate Anything on Earth: Advancing Open-Vocabulary Object Detection for Remote Sensing Community](https://mlsanity.com/paper/2408.09110/) (2024)
- [RemoteSAM: Towards Segment Anything for Earth Observation](https://mlsanity.com/paper/2505.18022/) (2025)
- [ROS-SAM: High-Quality Interactive Segmentation for Remote Sensing Moving Object](https://mlsanity.com/paper/2503.12006/) (2025)
- [InstructSAM: A Training-Free Framework for Instruction-Oriented Remote Sensing Object Recognition](https://mlsanity.com/paper/2505.15818/) (2025)
- [ZoRI: Towards Discriminative Zero-Shot Remote Sensing Instance Segmentation](https://mlsanity.com/paper/2412.12798/) (2024)

## Built on by (most cited first, 8 of 8)

- [UniGeoSeg: Towards Unified Open-World Segmentation for Geospatial Scenes](https://mlsanity.com/paper/2511.23332/) (2025)
- [RemoteAgent: Bridging Vague Human Intents and Earth Observation with RL-based Agentic MLLMs](https://mlsanity.com/paper/2604.07765/) (2026)
- [ConInfer: Context-Aware Inference for Training-Free Open-Vocabulary Remote Sensing Segmentation](https://mlsanity.com/paper/2603.29271/) (2026)
- [RemoteShield: Enable Robust Multimodal Large Language Models for Earth Observation](https://mlsanity.com/paper/2604.17243/) (2026)
- [A Dual-Branch Local-Global Framework for Cross-Resolution Land Cover Mapping](https://mlsanity.com/paper/2512.19990/) (2025)
- [Training-Free Open-Vocabulary Visual Grounding for Remote Sensing Images and Videos](https://mlsanity.com/paper/2606.16124/) (2026)
- [ExACT: Exemplar-Driven Calibrated Refinement for Training-Free Visual Grounding in Remote Sensing Images](https://mlsanity.com/paper/2606.28920/) (2026)
- [GeoSelect: Spatial-Program Execution for Training-Free Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2607.03869/) (2026)

## Abstract (arXiv)

Remote sensing visual grounding (RSVG) aims to localize objects in remote sensing images based on free-form natural language expressions. Existing approaches are typically constrained to closed-set vocabularies, limiting their applicability in open-world scenarios. While recent attempts to leverage generic foundation models for open-vocabulary RSVG, they overly rely on expensive high-quality datasets and time-consuming fine-tuning. To address these limitations, we propose RSVG-ZeroOV, a training-free framework that aims to explore the potential of frozen generic foundation models for zero-shot open-vocabulary RSVG. Specifically, RSVG-ZeroOV comprises three key stages: (i) Overview: We utilize a vision-language model (VLM) to obtain cross-attentionfootnote[1]In this paper, although decoder-only VLMs use self-attention over all tokens, we refer to the image-text interaction part as cross-attention to distinguish it from pure visual self-attention.maps that capture semantic correlations between text queries and visual regions. (ii) Focus: By leveraging the fine-grained modeling priors of a diffusion model (DM), we fill in gaps in structural and shape information of objects, which are often overlooked by VLM. (iii) Evolve: A simple yet effective attention evolution module is introduced to suppress irrelevant activations, yielding purified segmentation masks over the referred objects. Without cumbersome task-specific training, RSVG-ZeroOV offers an efficient and scalable solution. Extensive experiments demonstrate that the proposed framework consistently outperforms existing weakly-supervised and zero-shot methods.