# Unsupervised Hashing with Semantic Concept Mining

arXiv: https://arxiv.org/abs/2209.11475 · HTML: https://mlsanity.com/paper/2209.11475/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 64 references that are in the map.
- First posted on arXiv 23 Sep 2022.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text](https://mlsanity.com/paper/2104.11178/) (2021)
- [HashNet: Deep Learning to Hash by Continuation](https://mlsanity.com/paper/1702.00758/) (2017)
- [Central Similarity Quantization for Efficient Image and Video Retrieval](https://mlsanity.com/paper/1908.00347/) (2019)
- [Feature Learning based Deep Supervised Hashing with Pairwise Labels](https://mlsanity.com/paper/1511.03855/) (2015)
- [A Survey on Deep Hashing Methods](https://mlsanity.com/paper/2003.03369/) (2020)
- [Supervised Discrete Hashing](https://mlsanity.com/paper/1503.01557/) (2015)
- [Learning to Hash for Indexing Big Data - A Survey](https://mlsanity.com/paper/1509.05472/) (2015)
- [Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrieval](https://mlsanity.com/paper/1804.01223/) (2018)
- [DistillHash: Unsupervised Deep Hashing by Distilling Data Pairs](https://mlsanity.com/paper/1905.03465/) (2019)
- [Auto-Encoding Twin-Bottleneck Hashing](https://mlsanity.com/paper/2002.11930/) (2020)
- [Unsupervised Hashing with Contrastive Information Bottleneck](https://mlsanity.com/paper/2105.06138/) (2021)
- [Binary Generative Adversarial Networks for Image Retrieval](https://mlsanity.com/paper/1708.04150/) (2017)
- [CIMON: Towards High-quality Hash Codes](https://mlsanity.com/paper/2010.07804/) (2020)
- [Circulant Binary Embedding](https://mlsanity.com/paper/1405.3162/) (2014)
- [Deep Cross-Modal Hashing with Hashing Functions and Unified Hash Codes Jointly Learning](https://mlsanity.com/paper/1907.12490/) (2019)
- [Unsupervised Triplet Hashing for Fast Image Retrieval](https://mlsanity.com/paper/1702.08798/) (2017)
- [Shared Predictive Cross-Modal Deep Quantization](https://mlsanity.com/paper/1904.07488/) (2019)
- [Deep Spherical Quantization for Image Search](https://mlsanity.com/paper/1906.02865/) (2019)

## Built on by (most cited first, 5 of 5)

- [Hyperbolic Hierarchical Contrastive Hashing](https://mlsanity.com/paper/2212.08904/) (2022)
- [Multimodal Reasoning Agent for Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2505.19952/) (2025)
- [MLLM-Guided VLM Fine-Tuning with Joint Inference for Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2505.19707/) (2025)
- [Level of agreement between emotions generated by Artificial Intelligence and human evaluation: a methodological proposal](https://mlsanity.com/paper/2410.08332/) (2024)
- [Near-Oracle KV Selection via Pre-hoc Sparsity for Long-Context Inference](https://mlsanity.com/paper/2602.08329/) (2026)

## Abstract (arXiv)

Recently, to improve the unsupervised image retrieval performance, plenty of unsupervised hashing methods have been proposed by designing a semantic similarity matrix, which is based on the similarities between image features extracted by a pre-trained CNN model. However, most of these methods tend to ignore high-level abstract semantic concepts contained in images. Intuitively, concepts play an important role in calculating the similarity among images. In real-world scenarios, each image is associated with some concepts, and the similarity between two images will be larger if they share more identical concepts. Inspired by the above intuition, in this work, we propose a novel Unsupervised Hashing with Semantic Concept Mining, called UHSCM, which leverages a VLP model to construct a high-quality similarity matrix. Specifically, a set of randomly chosen concepts is first collected. Then, by employing a vision-language pretraining (VLP) model with the prompt engineering which has shown strong power in visual representation learning, the set of concepts is denoised according to the training images. Next, the proposed method UHSCM applies the VLP model with prompting again to mine the concept distribution of each image and construct a high-quality semantic similarity matrix based on the mined concept distributions. Finally, with the semantic similarity matrix as guiding information, a novel hashing loss with a modified contrastive loss based regularization item is proposed to optimize the hashing network. Extensive experiments on three benchmark datasets show that the proposed method outperforms the state-of-the-art baselines in the image retrieval task.