VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing

Satellite imagery differs from natural images in viewpoint, resolution, scale variation, and the prevalence of small objects -- demanding both region-level spatial reasoning and holistic scene understanding. Existing remote-sensing approaches are fragmented: dual-encoder retrieval models scale well but cannot interleave modalities, whereas generative assistants support grounding, yet are inefficient for retrieval. Benchmarks mirror this split: interleaved evaluations mainly target generative assistants, while cross-modal retrieval benchmarks target dual encoders. To bridge this gap, we introduce RSMEB, a unified remote sensing benchmark that evaluates cross-modal and interleaved retrieval across 21 tasks under a single ranking protocol, enabling comprehensive comparison of retrieval models on region- and geo-aware capabilities as well as conventional retrieval. As a strong reference baseline, we present VLM2GeoVec, an instruction-conditioned, single-encoder interleaving formulation tailored to remote sensing that packs image, text, bounding-box, and geo-coordinate tokens into one sequence and learns a unified embedding via contrastive training. Across RSMEB, VLM2GeoVec achieves 26.6% P@1 in region-caption retrieval (+25 percentage points), 32.5% in referring-expression retrieval (+19), and 17.8% in semantic geo-aware retrieval (>3× prior best), while remaining competitive in conventional scene classification and text--image retrieval in zero-shot settings. Together, the proposed suite and reference baseline standardize evaluation and deliver a unified embedder for scalable retrieval and region-/geo-aware grounding. The code, the model checkpoints, and the data are available at https://github.com/emasa/VLM2GeoVec.