# Semantics-Consistent Representation Learning for Remote Sensing Image-Voice Retrieval

arXiv: https://arxiv.org/abs/2103.05302 · HTML: https://mlsanity.com/paper/2103.05302/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Mar 2021.

## Built on by (most cited first, 8 of 8)

- [RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Data](https://mlsanity.com/paper/2210.12634/) (2022)
- [Parameter-Efficient Transfer Learning for Remote Sensing Image-Text Retrieval](https://mlsanity.com/paper/2308.12509/) (2023)
- [Learning Tri-modal Embeddings for Zero-Shot Soundscape Mapping](https://mlsanity.com/paper/2309.10667/) (2023)
- [Deep Unsupervised Contrastive Hashing for Large-Scale Cross-Modal Text-Image Retrieval in Remote Sensing](https://mlsanity.com/paper/2201.08125/) (2022)
- [Unsupervised Contrastive Hashing for Cross-Modal Retrieval in Remote Sensing](https://mlsanity.com/paper/2204.08707/) (2022)
- [Representation Discrepancy Bridging Method for Remote Sensing Image-Text Retrieval](https://mlsanity.com/paper/2505.16756/) (2025)
- [Braille-to-Speech Generator: Audio Generation Based on Joint Fine-Tuning of CLIP and Fastspeech2](https://mlsanity.com/paper/2407.14212/) (2024)
- [A Resource-Efficient Training Framework for Remote Sensing Text--Image Retrieval](https://mlsanity.com/paper/2501.10638/) (2025)

## Abstract (arXiv)

With the development of earth observation technology, massive amounts of remote sensing (RS) images are acquired. To find useful information from these images, cross-modal RS image-voice retrieval provides a new insight. This paper aims to study the task of RS image-voice retrieval so as to search effective information from massive amounts of RS data. Existing methods for RS image-voice retrieval rely primarily on the pairwise relationship to narrow the heterogeneous semantic gap between images and voices. However, apart from the pairwise relationship included in the datasets, the intra-modality and non-paired inter-modality relationships should also be taken into account simultaneously, since the semantic consistency among non-paired representations plays an important role in the RS image-voice retrieval task. Inspired by this, a semantics-consistent representation learning (SCRL) method is proposed for RS image-voice retrieval. The main novelty is that the proposed method takes the pairwise, intra-modality, and non-paired inter-modality relationships into account simultaneously, thereby improving the semantic consistency of the learned representations for the RS image-voice retrieval. The proposed SCRL method consists of two main steps: 1) semantics encoding and 2) semantics-consistent representation learning. Firstly, an image encoding network is adopted to extract high-level image features with a transfer learning strategy, and a voice encoding network with dilated convolution is devised to obtain high-level voice features. Secondly, a consistent representation space is conducted by modeling the three kinds of relationships to narrow the heterogeneous semantic gap and learn semantics-consistent representations across two modalities. Extensive experimental results on three challenging RS image-voice datasets show the effectiveness of the proposed method.