# Lightweight Contrastive Distilled Hashing for Online Cross-modal Retrieval

arXiv: https://arxiv.org/abs/2502.19751 · HTML: https://mlsanity.com/paper/2502.19751/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal distillation, one of 12 directions in Knowledge distillation.
- Builds on 8 of its 23 references that are in the map.
- First posted on arXiv 27 Feb 2025.

## Placed in

- Knowledge distillation / [Multimodal distillation](https://mlsanity.com/distill/multimodal-distillation/)

## Builds on (most cited first, 8 of 8)

- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)
- [A Comprehensive Survey on Cross-modal Retrieval](https://mlsanity.com/paper/1607.06215/) (2016)
- [Deep Cross-Modal Hashing](https://mlsanity.com/paper/1602.02255/) (2016)
- [Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrieval](https://mlsanity.com/paper/1804.01223/) (2018)
- [Creating Something from Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing](https://mlsanity.com/paper/2004.00280/) (2020)
- [Deep Multi-View Subspace Clustering with Anchor Graph](https://mlsanity.com/paper/2305.06939/) (2023)
- [Supervised Online Hashing via Hadamard Codebook Learning](https://mlsanity.com/paper/1905.03694/) (2019)
- [Deep Cross-Modal Hashing with Hashing Functions and Unified Hash Codes Jointly Learning](https://mlsanity.com/paper/1907.12490/) (2019)

## Abstract (arXiv)

Deep online cross-modal hashing has gained much attention from researchers recently, as its promising applications with low storage requirement, fast retrieval efficiency and cross modality adaptive, etc. However, there still exists some technical hurdles that hinder its applications, e.g., 1) how to extract the coexistent semantic relevance of cross-modal data, 2) how to achieve competitive performance when handling the real time data streams, 3) how to transfer the knowledge learned from offline to online training in a lightweight manner. To address these problems, this paper proposes a lightweight contrastive distilled hashing (LCDH) for cross-modal retrieval, by innovatively bridging the offline and online cross-modal hashing by similarity matrix approximation in a knowledge distillation framework. Specifically, in the teacher network, LCDH first extracts the cross-modal features by the contrastive language-image pre-training (CLIP), which are further fed into an attention module for representation enhancement after feature fusion. Then, the output of the attention module is fed into a FC layer to obtain hash codes for aligning the sizes of similarity matrices for online and offline training. In the student network, LCDH extracts the visual and textual features by lightweight models, and then the features are fed into a FC layer to generate binary codes. Finally, by approximating the similarity matrices, the performance of online hashing in the lightweight student network can be enhanced by the supervision of coexistent semantic relevance that is distilled from the teacher network. Experimental results on three widely used datasets demonstrate that LCDH outperforms some state-of-the-art methods.