# Self-Supervised Video Hashing with Hierarchical Binary Auto-encoder

arXiv: https://arxiv.org/abs/1802.02305 · HTML: https://mlsanity.com/paper/1802.02305/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Feb 2018.

## Built on by (most cited first, 30 of 34)

- [Synthesizing Informative Training Samples with GAN](https://mlsanity.com/paper/2204.07513/) (2022)
- [Multi-level Feature Learning for Contrastive Multi-view Clustering](https://mlsanity.com/paper/2106.11193/) (2021)
- [Long-Short Temporal Contrastive Learning of Video Transformers](https://mlsanity.com/paper/2106.09212/) (2021)
- [Deep Incomplete Multi-view Clustering with Cross-view Partial Sample and Prototype Alignment](https://mlsanity.com/paper/2303.15689/) (2023)
- [GCFAgg: Global and Cross-view Feature Aggregation for Multi-view Clustering](https://mlsanity.com/paper/2305.06799/) (2023)
- [A Review of Deep Learning for Video Captioning](https://mlsanity.com/paper/2304.11431/) (2023)
- [Artificial Intelligence for the Metaverse: A Survey](https://mlsanity.com/paper/2202.10336/) (2022)
- [DealMVC: Dual Contrastive Calibration for Multi-view Clustering](https://mlsanity.com/paper/2308.09000/) (2023)
- [DnS: Distill-and-Select for Efficient and Accurate Video Indexing and Retrieval](https://mlsanity.com/paper/2106.13266/) (2021)
- [SCVRL: Shuffled Contrastive Video Representation Learning](https://mlsanity.com/paper/2205.11710/) (2022)
- [Contrastive Masked Autoencoders for Self-Supervised Video Hashing](https://mlsanity.com/paper/2211.11210/) (2022)
- [Self-Supervised Video Similarity Learning](https://mlsanity.com/paper/2304.03378/) (2023)
- [One-shot Scene Graph Generation](https://mlsanity.com/paper/2202.10824/) (2022)
- [Decomposing Normal and Abnormal Features of Medical Images into Discrete Latent Codes for Content-Based Image Retrieval](https://mlsanity.com/paper/2103.12328/) (2021)
- [Self-supervised Video Retrieval Transformer Network](https://mlsanity.com/paper/2104.07993/) (2021)
- [3D-CSL: self-supervised 3D context similarity learning for Near-Duplicate Video Retrieval](https://mlsanity.com/paper/2211.05352/) (2022)
- [Dual-Stream Knowledge-Preserving Hashing for Unsupervised Video Retrieval](https://mlsanity.com/paper/2310.08009/) (2023)
- [Relation Regularized Scene Graph Generation](https://mlsanity.com/paper/2202.10826/) (2022)
- [Asymmetric double-winged multi-view clustering network for exploring Diverse and Consistent Information](https://mlsanity.com/paper/2309.00474/) (2023)
- [CHAIN: Exploring Global-Local Spatio-Temporal Information for Improved Self-Supervised Video Hashing](https://mlsanity.com/paper/2310.18926/) (2023)
- [Efficient Self-Supervised Video Hashing with Selective State Spaces](https://mlsanity.com/paper/2412.14518/) (2024)
- [Trusted Mamba Contrastive Network for Multi-View Clustering](https://mlsanity.com/paper/2412.16487/) (2024)
- [AutoSSVH: Exploring Automated Frame Sampling for Efficient Self-Supervised Video Hashing](https://mlsanity.com/paper/2504.03587/) (2025)
- [Live Laparoscopic Video Retrieval with Compressed Uncertainty](https://mlsanity.com/paper/2203.04301/) (2022)
- [Informative Scene Graph Generation via Debiasing](https://mlsanity.com/paper/2308.05286/) (2023)
- [Spiking Music: Audio Compression with Event Based Auto-encoders](https://mlsanity.com/paper/2402.01571/) (2024)
- [HyBNN and FedHyBNN: (Federated) Hybrid Binary Neural Networks](https://mlsanity.com/paper/2205.09839/) (2022)
- [Towards Generalized Multi-stage Clustering: Multi-view Self-distillation](https://mlsanity.com/paper/2310.18890/) (2023)
- [Query by Activity Video in the Wild](https://mlsanity.com/paper/2311.13895/) (2023)
- [Enhancing Video Understanding: Deep Neural Networks for Spatiotemporal Analysis](https://mlsanity.com/paper/2502.07277/) (2025)

## Abstract (arXiv)

Existing video hash functions are built on three isolated stages: frame pooling, relaxed learning, and binarization, which have not adequately explored the temporal order of video frames in a joint binary optimization model, resulting in severe information loss. In this paper, we propose a novel unsupervised video hashing framework dubbed Self-Supervised Video Hashing (SSVH), that is able to capture the temporal nature of videos in an end-to-end learning-to-hash fashion. We specifically address two central problems: 1) how to design an encoder-decoder architecture to generate binary codes for videos; and 2) how to equip the binary codes with the ability of accurate video retrieval. We design a hierarchical binary autoencoder to model the temporal dependencies in videos with multiple granularities, and embed the videos into binary codes with less computations than the stacked architecture. Then, we encourage the binary codes to simultaneously reconstruct the visual content and neighborhood structure of the videos. Experiments on two real-world datasets (FCVID and YFCC) show that our SSVH method can significantly outperform the state-of-the-art methods and achieve the currently best performance on the task of unsupervised video retrieval.