# Efficient Cross-Modal Retrieval via Deep Binary Hashing and Quantization

arXiv: https://arxiv.org/abs/2202.10232 · HTML: https://mlsanity.com/paper/2202.10232/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 50 references that are in the map.
- First posted on arXiv 15 Feb 2022.

## Builds on (most cited first, 14 of 14)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Deep Cross-Modal Hashing](https://mlsanity.com/paper/1602.02255/) (2016)
- [Supervised Discrete Hashing](https://mlsanity.com/paper/1503.01557/) (2015)
- [Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrieval](https://mlsanity.com/paper/1804.01223/) (2018)
- [Creating Something from Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing](https://mlsanity.com/paper/2004.00280/) (2020)
- [Unsupervised Generative Adversarial Cross-modal Hashing](https://mlsanity.com/paper/1712.00358/) (2017)
- [Multimodal music information processing and retrieval: survey and future challenges](https://mlsanity.com/paper/1902.05347/) (2019)
- [Composite Correlation Quantization for Efficient Multimodal Retrieval](https://mlsanity.com/paper/1504.04818/) (2015)
- [Deep Binary Reconstruction for Cross-modal Hashing](https://mlsanity.com/paper/1708.05127/) (2017)
- [Coupled CycleGAN: Unsupervised Hashing Network for Cross-Modal Retrieval](https://mlsanity.com/paper/1903.02149/) (2019)
- [Shared Predictive Cross-Modal Deep Quantization](https://mlsanity.com/paper/1904.07488/) (2019)
- [Correlation Hashing Network for Efficient Cross-Modal Retrieval](https://mlsanity.com/paper/1602.06697/) (2016)
- [Collaborative Quantization for Cross-Modal Similarity Search](https://mlsanity.com/paper/1902.00623/) (2019)

## Abstract (arXiv)

Cross-modal retrieval aims to search for data with similar semantic meanings across different content modalities. However, cross-modal retrieval requires huge amounts of storage and retrieval time since it needs to process data in multiple modalities. Existing works focused on learning single-source compact features such as binary hash codes that preserve similarities between different modalities. In this work, we propose a jointly learned deep hashing and quantization network (HQ) for cross-modal retrieval. We simultaneously learn binary hash codes and quantization codes to preserve semantic information in multiple modalities by an end-to-end deep learning architecture. At the retrieval step, binary hashing is used to retrieve a subset of items from the search space, then quantization is used to re-rank the retrieved items. We theoretically and empirically show that this two-stage retrieval approach provides faster retrieval results while preserving accuracy. Experimental results on the NUS-WIDE, MIR-Flickr, and Amazon datasets demonstrate that HQ achieves boosts of more than 7% in precision compared to supervised neural network-based compact coding models.