# CLIP Multi-modal Hashing for Multimedia Retrieval

arXiv: https://arxiv.org/abs/2410.07783 · HTML: https://mlsanity.com/paper/2410.07783/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 33 references that are in the map.
- First posted on arXiv 10 Oct 2024.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Learnable pooling with Context Gating for video classification](https://mlsanity.com/paper/1706.06905/) (2017)
- [Deep Metric Multi-View Hashing for Multimedia Retrieval](https://mlsanity.com/paper/2304.06358/) (2023)
- [Central Similarity Multi-View Hashing for Multimedia Retrieval](https://mlsanity.com/paper/2308.13774/) (2023)
- [Adaptive Confidence Multi-View Hashing for Multimedia Retrieval](https://mlsanity.com/paper/2312.07327/) (2023)

## Built on by (most cited first, 2 of 2)

- [Beyond Gaussian Bottlenecks: Topologically Aligned Encoding of Vision-Transformer Feature Spaces](https://mlsanity.com/paper/2604.28122/) (2026)
- [DSCH-Loss: A Dynamic Semantic Channel Objective for Deep Semantic Hashing](https://mlsanity.com/paper/2607.24567/) (2026)

## Abstract (arXiv)

Multi-modal hashing methods are widely used in multimedia retrieval, which can fuse multi-source data to generate binary hash code. However, the individual backbone networks have limited feature expression capabilities and are not jointly pre-trained on large-scale unsupervised multi-modal data, resulting in low retrieval accuracy. To address this issue, we propose a novel CLIP Multi-modal Hashing (CLIPMH) method. Our method employs the CLIP framework to extract both text and vision features and then fuses them to generate hash code. Due to enhancement on each modal feature, our method has great improvement in the retrieval performance of multi-modal hashing methods. Compared with state-of-the-art unsupervised and supervised multi-modal hashing methods, experiments reveal that the proposed CLIPMH can significantly improve performance (a maximum increase of 8.38% in mAP).