# Dual-Stream Knowledge-Preserving Hashing for Unsupervised Video Retrieval

arXiv: https://arxiv.org/abs/2310.08009 · HTML: https://mlsanity.com/paper/2310.08009/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 61 references that are in the map.
- First posted on arXiv 12 Oct 2023.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Unsupervised Learning of Video Representations using LSTMs](https://mlsanity.com/paper/1502.04681/) (2015)
- [Multi-modal Transformer for Video Retrieval](https://mlsanity.com/paper/2007.10639/) (2020)
- [Beyond Short Snippets: Deep Networks for Video Classification](https://mlsanity.com/paper/1503.08909/) (2015)
- [HashNet: Deep Learning to Hash by Continuation](https://mlsanity.com/paper/1702.00758/) (2017)
- [Cross-Modal and Hierarchical Modeling of Video and Text](https://mlsanity.com/paper/1810.07212/) (2018)
- [Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval](https://mlsanity.com/paper/2007.12163/) (2020)
- [From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network](https://mlsanity.com/paper/2108.09661/) (2021)
- [Domain-aware Visual Bias Eliminating for Generalized Zero-Shot Learning](https://mlsanity.com/paper/2003.13261/) (2020)
- [ContourNet: Taking a Further Step toward Accurate Arbitrary-shaped Scene Text Detection](https://mlsanity.com/paper/2004.04940/) (2020)
- [Label Refinery: Improving ImageNet Classification through Label Progression](https://mlsanity.com/paper/1805.02641/) (2018)
- [Exploiting Feature and Class Relationships in Video Categorization with Regularized Deep Neural Networks](https://mlsanity.com/paper/1502.07209/) (2015)
- [Semantic-Aware Knowledge Preservation for Zero-Shot Sketch-Based Image Retrieval](https://mlsanity.com/paper/1904.03208/) (2019)
- [DiVA: Diverse Visual Feature Aggregation for Deep Metric Learning](https://mlsanity.com/paper/2004.13458/) (2020)
- [Self-Supervised Video Hashing with Hierarchical Binary Auto-encoder](https://mlsanity.com/paper/1802.02305/) (2018)
- [DistillHash: Unsupervised Deep Hashing by Distilling Data Pairs](https://mlsanity.com/paper/1905.03465/) (2019)
- [Auto-Encoding Twin-Bottleneck Hashing](https://mlsanity.com/paper/2002.11930/) (2020)
- [Unsupervised Hashing with Contrastive Information Bottleneck](https://mlsanity.com/paper/2105.06138/) (2021)
- [End-to-End Video Text Spotting with Transformer](https://mlsanity.com/paper/2203.10539/) (2022)
- [ExchNet: A Unified Hashing Network for Large-Scale Fine-Grained Image Retrieval](https://mlsanity.com/paper/2008.01369/) (2020)
- [Graph Wasserstein Correlation Analysis for Movie Retrieval](https://mlsanity.com/paper/2008.02648/) (2020)

## Built on by (most cited first, 4 of 4)

- [Act As You Wish: Fine-Grained Control of Motion Diffusion Model with Hierarchical Semantic Graphs](https://mlsanity.com/paper/2311.01015/) (2023)
- [Towards Balanced Alignment: Modal-Enhanced Semantic Modeling for Video Moment Retrieval](https://mlsanity.com/paper/2312.12155/) (2023)
- [Efficient Self-Supervised Video Hashing with Selective State Spaces](https://mlsanity.com/paper/2412.14518/) (2024)
- [AutoSSVH: Exploring Automated Frame Sampling for Efficient Self-Supervised Video Hashing](https://mlsanity.com/paper/2504.03587/) (2025)

## Abstract (arXiv)

Unsupervised video hashing usually optimizes binary codes by learning to reconstruct input videos. Such reconstruction constraint spends much effort on frame-level temporal context changes without focusing on video-level global semantics that are more useful for retrieval. Hence, we address this problem by decomposing video information into reconstruction-dependent and semantic-dependent information, which disentangles the semantic extraction from reconstruction constraint. Specifically, we first design a simple dual-stream structure, including a temporal layer and a hash layer. Then, with the help of semantic similarity knowledge obtained from self-supervision, the hash layer learns to capture information for semantic retrieval, while the temporal layer learns to capture the information for reconstruction. In this way, the model naturally preserves the disentangled semantics into binary codes. Validated by comprehensive experiments, our method consistently outperforms the state-of-the-arts on three video benchmarks.