# HyP2 Loss: Beyond Hypersphere Metric Space for Multi-label Image Retrieval

arXiv: https://arxiv.org/abs/2208.06866 · HTML: https://mlsanity.com/paper/2208.06866/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 60 references that are in the map.
- First posted on arXiv 14 Aug 2022.

## Builds on (most cited first, 21 of 21)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [Proxy Anchor Loss for Deep Metric Learning](https://mlsanity.com/paper/2003.13911/) (2020)
- [SoftTriple Loss: Deep Metric Learning Without Triplet Sampling](https://mlsanity.com/paper/1909.05235/) (2019)
- [HashNet: Deep Learning to Hash by Continuation](https://mlsanity.com/paper/1702.00758/) (2017)
- [A Survey on Learning to Hash](https://mlsanity.com/paper/1606.00185/) (2016)
- [Smart Mining for Deep Metric Learning](https://mlsanity.com/paper/1704.01285/) (2017)
- [Central Similarity Quantization for Efficient Image and Video Retrieval](https://mlsanity.com/paper/1908.00347/) (2019)
- [Feature Learning based Deep Supervised Hashing with Pairwise Labels](https://mlsanity.com/paper/1511.03855/) (2015)
- [Simultaneous Feature Learning and Hash Coding with Deep Neural Networks](https://mlsanity.com/paper/1504.03410/) (2015)
- [One Loss for All: Deep Hashing with a Single Cosine Similarity based Learning Objective](https://mlsanity.com/paper/2109.14449/) (2021)
- [Supervised Discrete Hashing](https://mlsanity.com/paper/1503.01557/) (2015)
- [Learning to Hash for Indexing Big Data - A Survey](https://mlsanity.com/paper/1509.05472/) (2015)
- [Deep Supervised Hashing with Triplet Labels](https://mlsanity.com/paper/1612.03900/) (2016)
- [Deep Semantic Ranking Based Hashing for Multi-Label Image Retrieval](https://mlsanity.com/paper/1501.06272/) (2015)
- [Instance-Aware Hashing for Multi-Label Image Retrieval](https://mlsanity.com/paper/1603.03234/) (2016)
- [Rank-Consistency Deep Hashing for Scalable Multi-Label Image Search](https://mlsanity.com/paper/2102.01486/) (2021)
- [Two-pronged Strategy: Lightweight Augmented Graph Network Hashing for Scalable Image Retrieval](https://mlsanity.com/paper/2108.03914/) (2021)

## Built on by (most cited first, 3 of 3)

- [Deep Hashing via Householder Quantization](https://mlsanity.com/paper/2311.04207/) (2023)
- [Leveraging High-Resolution Features for Improved Deep Hashing-based Image Retrieval](https://mlsanity.com/paper/2403.13747/) (2024)
- [Sign-Symmetry Learning Rules are Robust Fine-Tuners](https://mlsanity.com/paper/2502.05925/) (2025)

## Abstract (arXiv)

Image retrieval has become an increasingly appealing technique with broad multimedia application prospects, where deep hashing serves as the dominant branch towards low storage and efficient retrieval. In this paper, we carried out in-depth investigations on metric learning in deep hashing for establishing a powerful metric space in multi-label scenarios, where the pair loss suffers high computational overhead and converge difficulty, while the proxy loss is theoretically incapable of expressing the profound label dependencies and exhibits conflicts in the constructed hypersphere space. To address the problems, we propose a novel metric learning framework with Hybrid Proxy-Pair Loss (HyP2 Loss) that constructs an expressive metric space with efficient training complexity w.r.t. the whole dataset. The proposed HyP2 Loss focuses on optimizing the hypersphere space by learnable proxies and excavating data-to-data correlations of irrelevant pairs, which integrates sufficient data correspondence of pair-based methods and high-efficiency of proxy-based methods. Extensive experiments on four standard multi-label benchmarks justify the proposed method outperforms the state-of-the-art, is robust among different hash bits and achieves significant performance gains with a faster, more stable convergence speed. Our code is available at https://github.com/JerryXu0129/HyP2-Loss.