# Contrastive Quantization with Code Memory for Unsupervised Image Retrieval

arXiv: https://arxiv.org/abs/2109.05205 · HTML: https://mlsanity.com/paper/2109.05205/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 52 references that are in the map.
- First posted on arXiv 11 Sep 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [Understanding the Behaviour of Contrastive Loss](https://mlsanity.com/paper/2012.09740/) (2020)
- [Debiased Contrastive Learning](https://mlsanity.com/paper/2007.00224/) (2020)
- [Cross-Batch Memory for Embedding Learning](https://mlsanity.com/paper/1912.06798/) (2019)
- [A Survey on Learning to Hash](https://mlsanity.com/paper/1606.00185/) (2016)
- [Central Similarity Quantization for Efficient Image and Video Retrieval](https://mlsanity.com/paper/1908.00347/) (2019)
- [DistillHash: Unsupervised Deep Hashing by Distilling Data Pairs](https://mlsanity.com/paper/1905.03465/) (2019)
- [Auto-Encoding Twin-Bottleneck Hashing](https://mlsanity.com/paper/2002.11930/) (2020)
- [Unsupervised Hashing with Contrastive Information Bottleneck](https://mlsanity.com/paper/2105.06138/) (2021)
- [Binary Generative Adversarial Networks for Image Retrieval](https://mlsanity.com/paper/1708.04150/) (2017)
- [Deep Unsupervised Image Hashing by Maximizing Bit Entropy](https://mlsanity.com/paper/2012.12334/) (2020)
- [Stochastic Generative Hashing](https://mlsanity.com/paper/1701.02815/) (2017)
- [Unsupervised Neural Quantization for Compressed-Domain Similarity Search](https://mlsanity.com/paper/1908.03883/) (2019)
- [BinGAN: Learning Compact Binary Descriptors with a Regularized GAN](https://mlsanity.com/paper/1806.06778/) (2018)
- [Unsupervised Triplet Hashing for Fast Image Retrieval](https://mlsanity.com/paper/1702.08798/) (2017)
- [Simultaneous Feature Aggregating and Hashing for Large-scale Image Search](https://mlsanity.com/paper/1704.00860/) (2017)

## Built on by (most cited first, 16 of 16)

- [Contrastive Conditional Latent Diffusion for Audio-visual Segmentation](https://mlsanity.com/paper/2307.16579/) (2023)
- [Residual Quantization with Implicit Neural Codebooks](https://mlsanity.com/paper/2401.14732/) (2024)
- [Backdoor Attack on Hash-based Image Retrieval via Clean-label Data Poisoning](https://mlsanity.com/paper/2109.08868/) (2021)
- [Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey](https://mlsanity.com/paper/2507.22920/) (2025)
- [NeuroHash: A Hyperdimensional Neuro-Symbolic Framework for Spatially-Aware Image Hashing and Retrieval](https://mlsanity.com/paper/2404.11025/) (2024)
- [A Simple Contrastive Framework Of Item Tokenization For Generative Recommendation](https://mlsanity.com/paper/2506.16683/) (2025)
- [A Survey of Historical Learning: Learning Models with Learning History](https://mlsanity.com/paper/2303.12992/) (2023)
- [Beyond Accuracy: Statistical Measures and Benchmark for Evaluation of Representation from Self-Supervised Learning](https://mlsanity.com/paper/2312.01118/) (2023)
- [Entity-Graph Enhanced Cross-Modal Pretraining for Instance-level Product Retrieval](https://mlsanity.com/paper/2206.08842/) (2022)
- [CoBooM: Codebook Guided Bootstrapping for Medical Image Representation Learning](https://mlsanity.com/paper/2408.04262/) (2024)
- [From Image Hashing to Scene Change Detection](https://mlsanity.com/paper/2605.12259/) (2026)
- [Collision-Resistant Single-Pass Method for Unsupervised Fine-Grained Image Hashing](https://mlsanity.com/paper/2605.18288/) (2026)
- [Self-Supervised Consistent Quantization for Fully Unsupervised Image Retrieval](https://mlsanity.com/paper/2206.09806/) (2022)
- [Towards Unseen Triples: Effective Text-Image-joint Learning for Scene Graph Generation](https://mlsanity.com/paper/2306.13420/) (2023)
- [Transformer-based Clipped Contrastive Quantization Learning for Unsupervised Image Retrieval](https://mlsanity.com/paper/2401.15362/) (2024)
- [Learn and Search: An Elegant Technique for Object Lookup using Contrastive Learning](https://mlsanity.com/paper/2403.07231/) (2024)

## Abstract (arXiv)

The high efficiency in computation and storage makes hashing (including binary hashing and quantization) a common strategy in large-scale retrieval systems. To alleviate the reliance on expensive annotations, unsupervised deep hashing becomes an important research problem. This paper provides a novel solution to unsupervised deep quantization, namely Contrastive Quantization with Code Memory (MeCoQ). Different from existing reconstruction-based strategies, we learn unsupervised binary descriptors by contrastive learning, which can better capture discriminative visual semantics. Besides, we uncover that codeword diversity regularization is critical to prevent contrastive learning-based quantization from model degeneration. Moreover, we introduce a novel quantization code memory module that boosts contrastive learning with lower feature drift than conventional feature memories. Extensive experiments on benchmark datasets show that MeCoQ outperforms state-of-the-art methods. Code and configurations are publicly available at https://github.com/gimpong/AAAI22-MeCoQ.