# A Novel Self-Supervised Cross-Modal Image Retrieval Method In Remote Sensing

arXiv: https://arxiv.org/abs/2202.11429 · HTML: https://mlsanity.com/paper/2202.11429/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 15 references that are in the map.
- First posted on arXiv 23 Feb 2022.

## Builds on (most cited first, 8 of 8)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [BigEarthNet-MM: A Large Scale Multi-Modal Multi-Label Benchmark Archive for Remote Sensing Image Classification and Retrieval](https://mlsanity.com/paper/2105.07921/) (2021)
- [Deep Learning for Image Search and Retrieval in Large Remote Sensing Archives](https://mlsanity.com/paper/2004.01613/) (2020)
- [CMIR-NET : A Deep Learning Based Model For Cross-Modal Retrieval In Remote Sensing](https://mlsanity.com/paper/1904.04794/) (2019)
- [Multi-Modal Mutual Information Maximization: A Novel Approach for Unsupervised Deep Cross-Modal Hashing](https://mlsanity.com/paper/2112.06489/) (2021)

## Built on by (most cited first, 10 of 10)

- [CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoders](https://mlsanity.com/paper/2311.00566/) (2023)
- [Learning Cross-view Visual Geo-localization without Ground Truth](https://mlsanity.com/paper/2403.12702/) (2024)
- [Exploring Masked Autoencoders for Sensor-Agnostic Image Retrieval in Remote Sensing](https://mlsanity.com/paper/2401.07782/) (2024)
- [SMARTIES: Spectrum-Aware Multi-Sensor Auto-Encoder for Remote Sensing Images](https://mlsanity.com/paper/2506.19585/) (2025)
- [Learning Across Decentralized Multi-Modal Remote Sensing Archives with Federated Learning](https://mlsanity.com/paper/2306.00792/) (2023)
- [Multi-Spectral Remote Sensing Image Retrieval Using Geospatial Foundation Models](https://mlsanity.com/paper/2403.02059/) (2024)
- [REJEPA: A Novel Joint-Embedding Predictive Architecture for Efficient Remote Sensing Image Retrieval](https://mlsanity.com/paper/2504.03169/) (2025)
- [CSMoE: An Efficient Remote Sensing Foundation Model with Soft Mixture-of-Experts](https://mlsanity.com/paper/2509.14104/) (2025)
- [A Multi-Modal Federated Learning Framework for Remote Sensing Image Classification](https://mlsanity.com/paper/2503.10262/) (2025)
- [CR-JEPA: Cross-Modal Joint-Embedding Predictive Learning for Remote Sensing Image Retrieval](https://mlsanity.com/paper/2606.00706/) (2026)

## Abstract (arXiv)

Due to the availability of multi-modal remote sensing (RS) image archives, one of the most important research topics is the development of cross-modal RS image retrieval (CM-RSIR) methods that search semantically similar images across different modalities. Existing CM-RSIR methods require the availability of a high quality and quantity of annotated training images. The collection of a sufficient number of reliable labeled images is time consuming, complex and costly in operational scenarios, and can significantly affect the final accuracy of CM-RSIR. In this paper, we introduce a novel self-supervised CM-RSIR method that aims to: i) model mutual-information between different modalities in a self-supervised manner; ii) retain the distributions of modal-specific feature spaces similar to each other; and iii) define the most similar images within each modality without requiring any annotated training image. To this end, we propose a novel objective including three loss functions that simultaneously: i) maximize mutual information of different modalities for inter-modal similarity preservation; ii) minimize the angular distance of multi-modal image tuples for the elimination of inter-modal discrepancies; and iii) increase cosine similarity of the most similar images within each modality for the characterization of intra-modal similarities. Experimental results show the effectiveness of the proposed method compared to state-of-the-art methods. The code of the proposed method is publicly available at https://git.tu-berlin.de/rsim/SS-CM-RSIR.