# Multi-Modal Mutual Information Maximization: A Novel Approach for Unsupervised Deep Cross-Modal Hashing

arXiv: https://arxiv.org/abs/2112.06489 · HTML: https://mlsanity.com/paper/2112.06489/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 93 references that are in the map.
- First posted on arXiv 13 Dec 2021.

## Builds on (most cited first, 30 of 36)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Contrastive Representation Distillation](https://mlsanity.com/paper/1910.10699/) (2019)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Learning Robust Representations via Multi-View Information Bottleneck](https://mlsanity.com/paper/2002.07017/) (2020)
- [REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models](https://mlsanity.com/paper/1703.07370/) (2017)
- [Wasserstein Contrastive Representation Distillation](https://mlsanity.com/paper/2012.08674/) (2020)
- [A Comprehensive Survey on Cross-modal Retrieval](https://mlsanity.com/paper/1607.06215/) (2016)
- [Deep Cross-Modal Hashing](https://mlsanity.com/paper/1602.02255/) (2016)
- [Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrieval](https://mlsanity.com/paper/1804.01223/) (2018)
- [Triplet-Based Deep Hashing Network for Cross-Modal Retrieval](https://mlsanity.com/paper/1904.02449/) (2019)
- [Creating Something from Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing](https://mlsanity.com/paper/2004.00280/) (2020)
- [Learning Representations by Maximizing Mutual Information in Variational Autoencoders](https://mlsanity.com/paper/1912.13361/) (2019)
- [Unsupervised Generative Adversarial Cross-modal Hashing](https://mlsanity.com/paper/1712.00358/) (2017)
- [Cycle-Consistent Deep Generative Hashing for Cross-Modal Retrieval](https://mlsanity.com/paper/1804.11013/) (2018)
- [Attribute-Guided Network for Cross-Modal Zero-Shot Hashing](https://mlsanity.com/paper/1802.01943/) (2018)
- [MIHash: Online Hashing with Mutual Information](https://mlsanity.com/paper/1703.08919/) (2017)
- [Deep Binary Reconstruction for Cross-modal Hashing](https://mlsanity.com/paper/1708.05127/) (2017)
- [Information Competing Process for Learning Diversified Representations](https://mlsanity.com/paper/1906.01288/) (2019)
- [Simultaneous Feature Aggregating and Hashing for Large-scale Image Search](https://mlsanity.com/paper/1704.00860/) (2017)

## Built on by (most cited first, 4 of 4)

- [A Novel Self-Supervised Cross-Modal Image Retrieval Method In Remote Sensing](https://mlsanity.com/paper/2202.11429/) (2022)
- [Zero-shot-Learning Cross-Modality Data Translation Through Mutual Information Guided Stochastic Diffusion](https://mlsanity.com/paper/2301.13743/) (2023)
- [Robust Multimodal Sentiment Analysis via Double Information Bottleneck](https://mlsanity.com/paper/2511.01444/) (2025)
- [Deep Lifelong Cross-modal Hashing](https://mlsanity.com/paper/2304.13357/) (2023)

## Abstract (arXiv)

In this paper, we adopt the maximizing mutual information (MI) approach to tackle the problem of unsupervised learning of binary hash codes for efficient cross-modal retrieval. We proposed a novel method, dubbed Cross-Modal Info-Max Hashing (CMIMH). First, to learn informative representations that can preserve both intra- and inter-modal similarities, we leverage the recent advances in estimating variational lower-bound of MI to maximize the MI between the binary representations and input features and between binary representations of different modalities. By jointly maximizing these MIs under the assumption that the binary representations are modelled by multivariate Bernoulli distributions, we can learn binary representations, which can preserve both intra- and inter-modal similarities, effectively in a mini-batch manner with gradient descent. Furthermore, we find out that trying to minimize the modality gap by learning similar binary representations for the same instance from different modalities could result in less informative representations. Hence, balancing between reducing the modality gap and losing modality-private information is important for the cross-modal retrieval tasks. Quantitative evaluations on standard benchmark datasets demonstrate that the proposed method consistently outperforms other state-of-the-art cross-modal retrieval methods.