mlsanityData to 7 Oct 2026

Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignment

arXiv 2312.06960 ↗First posted 12 Dec 2023cs.CV cs.LGcited by 71

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

We introduce a method to train vision-language models for remote-sensing images without using any textual annotations. Our key insight is to use co-located internet imagery taken on the ground as an intermediary for connecting remote-sensing images and language. Specifically, we train an image encoder for remote sensing images to align with the image encoder of CLIP using a large amount of paired internet and satellite images. Our unsupervised approach enables the training of a first-of-its-kind large-scale vision language model (VLM) for remote sensing images at two different resolutions. We show that these VLMs enable zero-shot, open-vocabulary image classification, retrieval, segmentation and visual question answering for satellite images. On each of these tasks, our VLM trained without textual annotations outperforms existing VLMs trained with supervision, with gains of up to 20% for classification and 80% for segmentation.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 53 references that are in the map. First posted on arXiv 12 Dec 2023.

Builds on

35 of 53 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023RSGPT: A Remote Sensing Vision Language Model and Benchmark28 Jul
  2. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensing19 Jun
  3. 2023Vision-Language Models in Remote Sensing: Current Progress and Future Trends9 May

Other references · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2023Segment Anythingcited by 8,158
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2020Supervised Contrastive Learningcited by 2,803
  10. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  11. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  12. 2021Grounded Language-Image Pre-trainingcited by 1,147
20 more
  1. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  2. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  3. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  4. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  5. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  6. 2021Image Segmentation Using Text and Image Promptscited by 509
  7. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 471
  8. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 402
  9. 2019BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understandingcited by 231
  10. 2021Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Datacited by 214
  11. 2020RSVQA: Visual Question Answering for Remote Sensing Datacited by 197
  12. 2017Exploring Models and Data for Remote Sensing Image Caption Generationcited by 195
  13. 2022Rethinking Visual Geo-localization for Large-Scale Applicationscited by 160
  14. 2022F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Modelscited by 149
  15. 2021Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implicationscited by 128
  16. 2021Coming Down to Earth: Satellite-to-Street View Synthesis for Geo-Localizationcited by 112
  17. 2022Deep Visual Geo-localization Benchmarkcited by 83
  18. 2015DeepSat - A Learning framework for Satellite Imagerycited by 37
  19. 2021Viewpoint Invariant Dense Matching for Visual Geolocalizationcited by 36
  20. 2021Seeing Through Clouds in Satellite Imagescited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 21; 2025: 29; 2026: 21.

71 papers.

Sort:

By area:

  1. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 291
  2. 2024SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understandingcited by 101
  3. 2024PANGAEA: A Global and Inclusive Benchmark for Geospatial Foundation Modelscited by 62
  4. 2024Locate Anything on Earth: Advancing Open-Vocabulary Object Detection for Remote Sensing Communitycited by 50
  5. 2025Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agentscited by 42
  6. 2024MetaEarth: A Generative Foundation Model for Global-Scale Remote Sensing Image Generationcited by 41
  7. 2024Towards Vision-Language Geo-Foundation Model: A Surveycited by 36
  8. 2024TaxaBind: A Unified Embedding Space for Ecological Applicationscited by 33
  9. 2024Major TOM: Expandable Datasets for Earth Observationcited by 31
  10. 2025Falcon: A Remote Sensing Vision-Language Foundation Model (Technical Report)cited by 28
  11. 2024GeoLLM-Engine: A Realistic Environment for Building Geospatial Copilotscited by 27
  12. 2025Multi-Agent Geospatial Copilots for Remote Sensing Workflowscited by 20
38 more
  1. 2025A Survey on Remote Sensing Foundation Models: From Vision to Multimodalitycited by 18
  2. 2025Vision-Language Modeling Meets Remote Sensing: Models, Datasets and Perspectivescited by 18
  3. 2025Update Your Transformer to the Latest Release: Re-Basin of Task Vectorscited by 17
  4. 2024Mind the Modality Gap: Towards a Remote Sensing Vision-Language Model via Cross-modal Alignmentcited by 15
  5. 2025GeoVLM-R1: Reinforcement Fine-Tuning for Improved Remote Sensing Reasoningcited by 15
  6. 2024WildSAT: Learning Satellite Image Representations from Wildlife Observationscited by 13
  7. 2026Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodalitycited by 12
  8. 2024GOMAA-Geo: GOal Modality Agnostic Active Geo-localizationcited by 11
  9. 2024Contrastive ground-level image and remote sensing pre-training improves representation learning for natural world imagerycited by 10
  10. 2025Quality-Driven Curation of Remote Sensing Vision-Language Data via Learned Scoring Modelscited by 10
  11. 2025Can Generative Geospatial Diffusion Models Excel as Discriminative Geospatial Foundation Models?cited by 10
  12. 2025Geo-OLM: Enabling Sustainable Earth Observation Studies with Cost-Efficient Open Language Models & State-Driven Workflowscited by 10
  13. 2024REO-VLM: Transforming VLM to Meet Regression Challenges in Earth Observationcited by 9
  14. 2024Enhancing Remote Sensing Vision-Language Models for Zero-Shot Scene Classificationcited by 8
  15. 2025Beyond the Visible: Multispectral Vision-Language Learning for Earth Observationcited by 7
  16. 2024ImageRAG: Enhancing Ultra High Resolution Remote Sensing Imagery Analysis with ImageRAGcited by 6
  17. 2025FLAIR-HUB: Large-scale Multimodal Dataset for Land Cover and Crop Mappingcited by 6
  18. 2025GeoMag: A Vision-Language Model for Pixel-level Fine-Grained Remote Sensing Image Parsingcited by 5
  19. 2025DescribeEarth: Describe Anything for Remote Sensing Imagescited by 5
  20. 2026EarthVL: A Progressive Earth Vision-Language Understanding and Generation Frameworkcited by 5
  21. 2024On the Generalizability of Foundation Models for Crop Type Mappingcited by 4
  22. 2025DGTRSD & DGTRS-CLIP: A Dual-Granularity Remote Sensing Image-Text Dataset and Vision Language Foundation Model for Alignmentcited by 4
  23. 2024Learnable Prompt for Few-Shot Semantic Segmentation in Remote Sensing Domaincited by 3
  24. 2025DiSciPLE: Learning Interpretable Programs for Scientific Visual Discoverycited by 3
  25. 2025UAV-VL-R1: Generalizing Vision-Language Models via Supervised Fine-Tuning and Multi-Stage GRPO for UAV Visual Reasoningcited by 3
  26. 2024Towards Knowledge Guided Pretraining Approaches for Multimodal Foundation Models: Applications in Remote Sensingcited by 2
  27. 2024Scale-Aware Recognition in Satellite Images under Resource Constraintscited by 2
  28. 2024Large Vision-Language Models for Remote Sensing Visual Question Answeringcited by 2
  29. 2025MONITRS: Multimodal Observations of Natural Incidents Through Remote Sensingcited by 2
  30. 2025Automated Wildfire Damage Assessment from Multi view Ground level Imagery Via Vision Language Modelscited by 2
  31. 2026Urban Socio-Semantic Segmentation with Vision-Language Reasoningcited by 2
  32. 2026GeoMeld: Toward Semantically Grounded Foundation Models for Remote Sensingcited by 2
  33. 2024Advancements in Visual Language Models for Remote Sensing: Datasets, Capabilities, and Enhancement Techniquescited by 1
  34. 2025SAR Object Detection with Self-Supervised Pretraining and Curriculum-Aware Samplingcited by 1
  35. 2025Habitat Classification from Ground-Level Imagery Using Deep Neural Networkscited by 1
  36. 2025Remote Sensing Image Intelligent Interpretation with the Language-Centered Perspective: Principles, Methods and Challengescited by 1
  37. 2025TimeSenCLIP: A Time Series Vision-Language Model for Remote Sensingcited by 1
  38. 2025ProM3E: Probabilistic Masked MultiModal Embedding Model for Ecologycited by 1

50 most cited of 71 All 71 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing
  2. 2024Pushing the Limits of Vision-Language Models in Remote Sensing without Human Annotations
  3. 2025Co-Training Vision Language Models for Remote Sensing Multi-task Learning
  4. 2026More with Less: a Large Scale Remote Sensing VLM with a Simple Recipe
  5. 2025Enhancing Remote Sensing Vision-Language Models Through MLLM and LLM-Based High-Quality Image-Text Dataset Generation
  6. 2025A Vision Centric Remote Sensing Benchmark
  7. 2024RSUniVLM: A Unified Vision Language Model for Remote Sensing via Granularity-oriented Mixture of Experts
  8. 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model