# Enhancing Remote Sensing Vision-Language Models for Zero-Shot Scene Classification

arXiv: https://arxiv.org/abs/2409.00698 · HTML: https://mlsanity.com/paper/2409.00698/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 25 of its 45 references that are in the map.
- First posted on arXiv 1 Sep 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Remote Sensing Image Scene Classification: Benchmark and State of the Art](https://mlsanity.com/paper/1703.00121/) (2017)
- [BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs](https://mlsanity.com/paper/2303.00915/) (2023)
- [RemoteCLIP: A Vision Language Foundation Model for Remote Sensing](https://mlsanity.com/paper/2306.11029/) (2023)
- [AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification](https://mlsanity.com/paper/1608.05167/) (2016)
- [A Baseline for Few-Shot Image Classification](https://mlsanity.com/paper/1909.02729/) (2019)
- [RSGPT: A Remote Sensing Vision Language Model and Benchmark](https://mlsanity.com/paper/2307.15266/) (2023)
- [EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domain](https://mlsanity.com/paper/2401.16822/) (2024)
- [Learning Visual Features from Large Weakly Supervised Data](https://mlsanity.com/paper/1511.02251/) (2015)
- [RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing](https://mlsanity.com/paper/2306.11300/) (2023)
- [LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model](https://mlsanity.com/paper/2402.02544/) (2024)
- [SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing](https://mlsanity.com/paper/2312.12856/) (2023)
- [Learning Visual Representations with Caption Annotations](https://mlsanity.com/paper/2008.01392/) (2020)
- [SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understanding](https://mlsanity.com/paper/2406.10100/) (2024)
- [PatternNet: A Benchmark Dataset for Performance Evaluation of Remote Sensing Image Retrieval](https://mlsanity.com/paper/1706.03424/) (2017)
- [Prototype Rectification for Few-Shot Learning](https://mlsanity.com/paper/1911.10713/) (2019)
- [Vision-Language Models in Remote Sensing: Current Progress and Future Trends](https://mlsanity.com/paper/2305.05726/) (2023)
- [Laplacian Regularized Few-Shot Learning](https://mlsanity.com/paper/2006.15486/) (2020)
- [Does CLIP Benefit Visual Question Answering in the Medical Domain as Much as it Does in the General Domain?](https://mlsanity.com/paper/2112.13906/) (2021)
- [Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignment](https://mlsanity.com/paper/2312.06960/) (2023)
- [On the test-time zero-shot generalization of vision-language models: Do we really need prompt learning?](https://mlsanity.com/paper/2405.02266/) (2024)
- [MLRSNet: A Multi-label High Spatial Resolution Remote Sensing Dataset for Semantic Scene Understanding](https://mlsanity.com/paper/2010.00243/) (2020)
- [Boosting Vision-Language Models with Transduction](https://mlsanity.com/paper/2406.01837/) (2024)
- [Transductive Zero-Shot and Few-Shot CLIP](https://mlsanity.com/paper/2405.18437/) (2024)
- [Streamlined Hybrid Annotation Framework using Scalable Codestream for Bandwidth-Restricted UAV Object Detection](https://mlsanity.com/paper/2402.04673/) (2024)

## Built on by (most cited first, 8 of 8)

- [Vision-Language Modeling Meets Remote Sensing: Models, Datasets and Perspectives](https://mlsanity.com/paper/2505.14361/) (2025)
- [Realistic Test-Time Adaptation of Vision-Language Models](https://mlsanity.com/paper/2501.03729/) (2025)
- [Online Gaussian Test-Time Adaptation of Vision-Language Models](https://mlsanity.com/paper/2501.04352/) (2025)
- [Survey of Multimodal Geospatial Foundation Models: Techniques, Applications, and Challenges](https://mlsanity.com/paper/2510.22964/) (2025)
- [Few-Shot Adaptation Benchmark for Remote Sensing Vision-Language Models](https://mlsanity.com/paper/2510.07135/) (2025)
- [Leveraging Prediction Entropy for Automatic Prompt Weighting in Zero-Shot Audio-Language Classification](https://mlsanity.com/paper/2601.05011/) (2026)
- [Locally Consistent Transductive Information Maximization for Few-Shot Remote Sensing Scene Classification](https://mlsanity.com/paper/2607.29192/) (2026)
- [Language-Aware Information Maximization for Transductive Few-Shot CLIP](https://mlsanity.com/paper/2509.00305/) (2025)

## Abstract (arXiv)

Vision-Language Models for remote sensing have shown promising uses thanks to their extensive pretraining. However, their conventional usage in zero-shot scene classification methods still involves dividing large images into patches and making independent predictions, i.e., inductive inference, thereby limiting their effectiveness by ignoring valuable contextual information. Our approach tackles this issue by utilizing initial predictions based on text prompting and patch affinity relationships from the image encoder to enhance zero-shot capabilities through transductive inference, all without the need for supervision and at a minor computational cost. Experiments on 10 remote sensing datasets with state-of-the-art Vision-Language Models demonstrate significant accuracy improvements over inductive zero-shot classification. Our source code is publicly available on Github: https://github.com/elkhouryk/RS-TransCLIP