# Creating Something from Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing

arXiv: https://arxiv.org/abs/2004.00280 · HTML: https://mlsanity.com/paper/2004.00280/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2020.

## Built on by (most cited first, 26 of 26)

- [EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation](https://mlsanity.com/paper/2111.12341/) (2021)
- [Domain-Agnostic Prior for Transfer Semantic Segmentation](https://mlsanity.com/paper/2204.02684/) (2022)
- [AdvPicker: Effectively Leveraging Unlabeled Data via Adversarial Discriminator for Cross-Lingual NER](https://mlsanity.com/paper/2106.02300/) (2021)
- [Generalized Knowledge Distillation via Relationship Matching](https://mlsanity.com/paper/2205.01915/) (2022)
- [Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipation](https://mlsanity.com/paper/2210.05991/) (2022)
- [Segment Any Events via Weighted Adaptation of Pivotal Tokens](https://mlsanity.com/paper/2312.16222/) (2023)
- [Multi-Modal Mutual Information Maximization: A Novel Approach for Unsupervised Deep Cross-Modal Hashing](https://mlsanity.com/paper/2112.06489/) (2021)
- [Synthetic data generation method for data-free knowledge distillation in regression neural networks](https://mlsanity.com/paper/2301.04338/) (2023)
- [uDistil-Whisper: Label-Free Data Filtering for Knowledge Distillation in Low-Data Regimes](https://mlsanity.com/paper/2407.01257/) (2024)
- [Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos](https://mlsanity.com/paper/2112.08117/) (2021)
- [Self Regulated Learning Mechanism for Data Efficient Knowledge Distillation](https://mlsanity.com/paper/2102.07125/) (2021)
- [Confidence Conditioned Knowledge Distillation](https://mlsanity.com/paper/2107.06993/) (2021)
- [Deep Supervised Information Bottleneck Hashing for Cross-modal Retrieval based Computer-aided Diagnosis](https://mlsanity.com/paper/2205.08365/) (2022)
- [Binary Representation via Jointly Personalized Sparse Hashing](https://mlsanity.com/paper/2208.14883/) (2022)
- [PromptHash: Affinity-Prompted Collaborative Cross-Modal Learning for Adaptive Hashing Retrieval](https://mlsanity.com/paper/2503.16064/) (2025)
- [Efficient Cross-Modal Retrieval via Deep Binary Hashing and Quantization](https://mlsanity.com/paper/2202.10232/) (2022)
- [Information-Theoretic Hashing for Zero-Shot Cross-Modal Retrieval](https://mlsanity.com/paper/2209.12491/) (2022)
- [360^circ High-Resolution Depth Estimation via Uncertainty-aware Structural Knowledge Transfer](https://mlsanity.com/paper/2304.07967/) (2023)
- [DeepAqua: Self-Supervised Semantic Segmentation of Wetland Surface Water Extent with SAR Images using Knowledge Distillation](https://mlsanity.com/paper/2305.01698/) (2023)
- [One-bit Supervision for Image Classification: Problem, Solution, and Beyond](https://mlsanity.com/paper/2311.15225/) (2023)
- [Distilling Vision-Language Pretraining for Efficient Cross-Modal Retrieval](https://mlsanity.com/paper/2405.14726/) (2024)
- [Information Extraction from Heterogeneous Documents without Ground Truth Labels using Synthetic Label Generation and Knowledge Distillation](https://mlsanity.com/paper/2411.14957/) (2024)
- [Lightweight Contrastive Distilled Hashing for Online Cross-modal Retrieval](https://mlsanity.com/paper/2502.19751/) (2025)
- [Semantic-Cohesive Knowledge Distillation for Deep Cross-modal Hashing](https://mlsanity.com/paper/2510.09664/) (2025)
- [Attribute-Prompted Kernel Hashing for Unsupervised Data-Efficient Cross-Modal Retrieval](https://mlsanity.com/paper/2607.00379/) (2026)
- [UMSS: Towards Unsupervised Multi-modal Semantic Segmentation](https://mlsanity.com/paper/2607.12372/) (2026)

## Abstract (arXiv)

In recent years, cross-modal hashing (CMH) has attracted increasing attentions, mainly because its potential ability of mapping contents from different modalities, especially in vision and language, into the same space, so that it becomes efficient in cross-modal data retrieval. There are two main frameworks for CMH, differing from each other in whether semantic supervision is required. Compared to the unsupervised methods, the supervised methods often enjoy more accurate results, but require much heavier labors in data annotation. In this paper, we propose a novel approach that enables guiding a supervised method using outputs produced by an unsupervised method. Specifically, we make use of teacher-student optimization for propagating knowledge. Experiments are performed on two popular CMH benchmarks, i.e., the MIRFlickr and NUS-WIDE datasets. Our approach outperforms all existing unsupervised methods by a large margin.