# Beyond Post-Quantization: Native Hash Learning with a Dedicated HASH Token

arXiv: https://arxiv.org/abs/2607.03328 · HTML: https://mlsanity.com/paper/2607.03328/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 43 references that are in the map.
- First posted on arXiv 3 Jul 2026.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [No Fuss Distance Metric Learning using Proxies](https://mlsanity.com/paper/1703.07464/) (2017)
- [Proxy Anchor Loss for Deep Metric Learning](https://mlsanity.com/paper/2003.13911/) (2020)
- [Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking](https://mlsanity.com/paper/1803.11285/) (2018)
- [HashNet: Deep Learning to Hash by Continuation](https://mlsanity.com/paper/1702.00758/) (2017)
- [Central Similarity Quantization for Efficient Image and Video Retrieval](https://mlsanity.com/paper/1908.00347/) (2019)
- [Feature Learning based Deep Supervised Hashing with Pairwise Labels](https://mlsanity.com/paper/1511.03855/) (2015)
- [One Loss for All: Deep Hashing with a Single Cosine Similarity based Learning Objective](https://mlsanity.com/paper/2109.14449/) (2021)
- [Asymmetric Deep Supervised Hashing](https://mlsanity.com/paper/1707.08325/) (2017)
- [TransHash: Transformer-based Hamming Hashing for Efficient Image Retrieval](https://mlsanity.com/paper/2105.01823/) (2021)
- [Deep Hashing with Hash Center Update for Efficient Image Retrieval](https://mlsanity.com/paper/2106.06269/) (2021)
- [Deep Hashing with Semantic Hash Centers for Image Retrieval](https://mlsanity.com/paper/2507.08404/) (2025)
- [HybridHash: Hybrid Convolutional and Self-Attention Deep Hashing for Image Retrieval](https://mlsanity.com/paper/2405.07524/) (2024)
- [MambaHash: Visual State Space Deep Hashing Model for Large-Scale Image Retrieval](https://mlsanity.com/paper/2506.16353/) (2025)
- [Codebook-Centric Deep Hashing: End-to-End Joint Learning of Semantic Hash Centers and Neural Hash Function](https://mlsanity.com/paper/2511.12162/) (2025)

## Abstract (arXiv)

Efficient large-scale image retrieval requires compact representations that preserve semantic similarity under fast Hamming-space search. Deep hashing is appealing, but most existing CNN- and ViT-based methods still follow a post-quantization paradigm, where continuous visual features are first learned and binary codes are then produced by a terminal hash projection or binarization operation. This late code generation creates a feature-to-code discrepancy between the continuously optimized representation space and the discrete Hamming space used for retrieval. To address this limitation, we propose HashViT, a Vision Transformer framework for native hash token learning. Instead of treating hashing as a terminal readout, HashViT introduces a dedicated HASH token that serves as a persistent, hash-oriented retrieval state inside the transformer. The HASH token is structurally decomposed into a Hash Register for direct binary code generation and a Semantic Workspace for preserving auxiliary continuous semantics. To enable effective workspace-to-register interaction, we further design a lightweight Hash Refinement Adapter that progressively refines the Hash Register across transformer layers. As a result, binary-oriented representations are formed through token evolution within the backbone, rather than being abruptly induced by an output-level projection. HashViT is optimized with a unified objective that combines learnable semantic center supervision, class-token similarity distillation, and quantization regularization, encouraging the HASH token to encode semantically structured and compact binary representations. Extensive experiments on three widely used benchmarks demonstrate that HashViT achieves state-of-the-art or highly competitive retrieval performance while preserving the efficiency of compact Hamming codes. Code is available at https://github.com/Xinze919/HashViT.