# Information-Theoretic Hashing for Zero-Shot Cross-Modal Retrieval

arXiv: https://arxiv.org/abs/2209.12491 · HTML: https://mlsanity.com/paper/2209.12491/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 56 references that are in the map.
- First posted on arXiv 26 Sep 2022.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Masked Feature Prediction for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2112.09133/) (2021)
- [TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech](https://mlsanity.com/paper/2007.06028/) (2020)
- [Learning a Deep Embedding Model for Zero-Shot Learning](https://mlsanity.com/paper/1611.05088/) (2016)
- [Cross-domain Correspondence Learning for Exemplar-based Image Translation](https://mlsanity.com/paper/2004.05571/) (2020)
- [Semantics-aware BERT for Language Understanding](https://mlsanity.com/paper/1909.02209/) (2019)
- [Semantic Image Synthesis via Adversarial Learning](https://mlsanity.com/paper/1707.06873/) (2017)
- [Cross Modal Retrieval with Querybank Normalisation](https://mlsanity.com/paper/2112.12777/) (2021)
- [A Comprehensive Survey on Cross-modal Retrieval](https://mlsanity.com/paper/1607.06215/) (2016)
- [HSVA: Hierarchical Semantic-Visual Adaptation for Zero-Shot Learning](https://mlsanity.com/paper/2109.15163/) (2021)
- [Deep Cross-Modal Hashing](https://mlsanity.com/paper/1602.02255/) (2016)
- [Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrieval](https://mlsanity.com/paper/1804.01223/) (2018)
- [Creating Something from Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing](https://mlsanity.com/paper/2004.00280/) (2020)
- [An Overview of Cross-media Retrieval: Concepts, Methodologies, Benchmarks and Challenges](https://mlsanity.com/paper/1704.02223/) (2017)
- [Comprehensive Graph-conditional Similarity Preserving Network for Unsupervised Cross-modal Hashing](https://mlsanity.com/paper/2012.13538/) (2020)
- [MHTN: Modal-adversarial Hybrid Transfer Network for Cross-modal Retrieval](https://mlsanity.com/paper/1708.04308/) (2017)
- [Cycle-Consistent Deep Generative Hashing for Cross-Modal Retrieval](https://mlsanity.com/paper/1804.11013/) (2018)
- [Attribute-Guided Network for Cross-Modal Zero-Shot Hashing](https://mlsanity.com/paper/1802.01943/) (2018)
- [Multivariate Extension of Matrix-based Renyi's α-order Entropy Functional](https://mlsanity.com/paper/1808.07912/) (2018)
- [Cross-modal Zero-shot Hashing](https://mlsanity.com/paper/1908.07388/) (2019)
- [PRI-VAE: Principle-of-Relevant-Information Variational Autoencoders](https://mlsanity.com/paper/2007.06503/) (2020)

## Abstract (arXiv)

Zero-shot cross-modal retrieval (ZS-CMR) deals with the retrieval problem among heterogenous data from unseen classes. Typically, to guarantee generalization, the pre-defined class embeddings from natural language processing (NLP) models are used to build a common space. In this paper, instead of using an extra NLP model to define a common space beforehand, we consider a totally different way to construct (or learn) a common hamming space from an information-theoretic perspective. We term our model the Information-Theoretic Hashing (ITH), which is composed of two cascading modules: an Adaptive Information Aggregation (AIA) module; and a Semantic Preserving Encoding (SPE) module. Specifically, our AIA module takes the inspiration from the Principle of Relevant Information (PRI) to construct a common space that adaptively aggregates the intrinsic semantics of different modalities of data and filters out redundant or irrelevant information. On the other hand, our SPE module further generates the hashing codes of different modalities by preserving the similarity of intrinsic semantics with the element-wise Kullback-Leibler (KL) divergence. A total correlation regularization term is also imposed to reduce the redundancy amongst different dimensions of hash codes. Sufficient experiments on three benchmark datasets demonstrate the superiority of the proposed ITH in ZS-CMR. Source code is available in the supplementary material.