# HIER: Metric Learning Beyond Class Labels via Hierarchical Regularization

arXiv: https://arxiv.org/abs/2212.14258 · HTML: https://mlsanity.com/paper/2212.14258/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 63 references that are in the map.
- First posted on arXiv 29 Dec 2022.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning](https://mlsanity.com/paper/1904.06627/) (2019)
- [No Fuss Distance Metric Learning using Proxies](https://mlsanity.com/paper/1703.07464/) (2017)
- [Hyperbolic Image Embeddings](https://mlsanity.com/paper/1904.02239/) (2019)
- [Representation Tradeoffs for Hyperbolic Embeddings](https://mlsanity.com/paper/1804.03329/) (2018)
- [Proxy Anchor Loss for Deep Metric Learning](https://mlsanity.com/paper/2003.13911/) (2020)
- [Learning Fine-grained Image Similarity with Deep Ranking](https://mlsanity.com/paper/1404.4661/) (2014)
- [Joint Detection and Identification Feature Learning for Person Search](https://mlsanity.com/paper/1604.01850/) (2016)
- [Beyond triplet loss: a deep quadruplet network for person re-identification](https://mlsanity.com/paper/1704.01719/) (2017)
- [SoftTriple Loss: Deep Metric Learning Without Triplet Sampling](https://mlsanity.com/paper/1909.05235/) (2019)
- [Cross-Batch Memory for Embedding Learning](https://mlsanity.com/paper/1912.06798/) (2019)
- [Deep Metric Learning with Hierarchical Triplet Loss](https://mlsanity.com/paper/1810.06951/) (2018)
- [Hyperbolic Vision Transformers: Combining Improvements in Metric Learning](https://mlsanity.com/paper/2203.10833/) (2022)
- [Learning to Compare Image Patches via Convolutional Neural Networks](https://mlsanity.com/paper/1504.03641/) (2015)
- [ProxyNCA++: Revisiting and Revitalizing Proxy Neighborhood Component Analysis](https://mlsanity.com/paper/2004.01113/) (2020)
- [Training Vision Transformers for Image Retrieval](https://mlsanity.com/paper/2102.05644/) (2021)
- [Ranked List Loss for Deep Metric Learning](https://mlsanity.com/paper/1903.03238/) (2019)
- [Transductive Episodic-Wise Adaptive Metric for Few-Shot Learning](https://mlsanity.com/paper/1910.02224/) (2019)
- [From Trees to Continuous Embeddings and Back: Hyperbolic Hierarchical Clustering](https://mlsanity.com/paper/2010.00402/) (2020)
- [A cost function for similarity-based hierarchical clustering](https://mlsanity.com/paper/1510.05043/) (2015)

## Built on by (most cited first, 26 of 26)

- [Hyperbolic Deep Learning in Computer Vision: A Survey](https://mlsanity.com/paper/2305.06611/) (2023)
- [ILIAS: Instance-Level Image retrieval At Scale](https://mlsanity.com/paper/2502.11748/) (2025)
- [Improving Visual Recognition with Hyperbolical Visual Hierarchy Mapping](https://mlsanity.com/paper/2404.00974/) (2024)
- [Learning Visual Hierarchies in Hyperbolic Space for Image Retrieval](https://mlsanity.com/paper/2411.17490/) (2024)
- [Threshold-Consistent Margin Loss for Open-World Deep Metric Learning](https://mlsanity.com/paper/2307.04047/) (2023)
- [Learning Semantic Proxies from Visual Prompts for Parameter-Efficient Fine-Tuning in Deep Metric Learning](https://mlsanity.com/paper/2402.02340/) (2024)
- [Globally Correlation-Aware Hard Negative Generation](https://mlsanity.com/paper/2411.13145/) (2024)
- [Anchor-aware Deep Metric Learning for Audio-visual Retrieval](https://mlsanity.com/paper/2404.13789/) (2024)
- [Robust Hyperbolic Learning with Curvature-Aware Optimization](https://mlsanity.com/paper/2405.13979/) (2024)
- [Potential Field Based Deep Metric Learning](https://mlsanity.com/paper/2405.18560/) (2024)
- [Learning Structured Representations by Embedding Class Hierarchy with Fast Optimal Transport](https://mlsanity.com/paper/2410.03052/) (2024)
- [Language-driven Fine-grained Retrieval](https://mlsanity.com/paper/2512.06255/) (2025)
- [Balanced Hyperbolic Embeddings Are Natural Out-of-Distribution Detectors](https://mlsanity.com/paper/2506.10146/) (2025)
- [LIHE: Linguistic Instance-Split Hyperbolic-Euclidean Framework for Generalized Weakly-Supervised Referring Expression Comprehension](https://mlsanity.com/paper/2511.12020/) (2025)
- [Head Similarity: Modeling Structured Whole-Head Appearance Beyond Face Recognition](https://mlsanity.com/paper/2605.07766/) (2026)
- [Categorical Knowledge Fused Recognition: Fusing Hierarchical Knowledge with Image Classification through Aligning and Deep Metric Learning](https://mlsanity.com/paper/2407.20600/) (2024)
- [Realigned Softmax Warping for Deep Metric Learning](https://mlsanity.com/paper/2408.15656/) (2024)
- [HypeVPR: Exploring Hyperbolic Space for Perspective to Equirectangular Visual Place Recognition](https://mlsanity.com/paper/2506.04764/) (2025)
- [Combined Hyperbolic and Euclidean Soft Triple Loss Beyond the Single Space Deep Metric Learning](https://mlsanity.com/paper/2510.05643/) (2025)
- [BLENDER: Blended Text Embeddings and Diffusion Residuals for Intra-Class Image Synthesis in Deep Metric Learning](https://mlsanity.com/paper/2601.20246/) (2026)
- [LAtte: Hyperbolic Lorentz Attention for Cross-Subject EEG Classification](https://mlsanity.com/paper/2603.10881/) (2026)
- [Hyp2Former: Hierarchy-Aware Hyperbolic Embeddings for Open-Set Panoptic Segmentation](https://mlsanity.com/paper/2605.02580/) (2026)
- [Learning to Align Generative Appearance Priors for Fine-grained Image Retrieval](https://mlsanity.com/paper/2605.09859/) (2026)
- [Beyond Scalar Distances: Semantic Attribute Gradients from Frozen MLLMs for Visual Embeddings](https://mlsanity.com/paper/2606.15134/) (2026)
- [CouCE: A Unified Causal Framework for Debiased Deep Metric Learning](https://mlsanity.com/paper/2606.30365/) (2026)
- [What Drives Hierarchy-Aware Image Retrieval? Taxonomy Alignment, Objective Choice, and Geometry](https://mlsanity.com/paper/2609.25638/) (2026)

## Abstract (arXiv)

Supervision for metric learning has long been given in the form of equivalence between human-labeled classes. Although this type of supervision has been a basis of metric learning for decades, we argue that it hinders further advances in the field. In this regard, we propose a new regularization method, dubbed HIER, to discover the latent semantic hierarchy of training data, and to deploy the hierarchy to provide richer and more fine-grained supervision than inter-class separability induced by common metric learning losses.HIER achieves this goal with no annotation for the semantic hierarchy but by learning hierarchical proxies in hyperbolic spaces. The hierarchical proxies are learnable parameters, and each of them is trained to serve as an ancestor of a group of data or other proxies to approximate the semantic hierarchy among them. HIER deals with the proxies along with data in hyperbolic space since the geometric properties of the space are well-suited to represent their hierarchical structure. The efficacy of HIER is evaluated on four standard benchmarks, where it consistently improved the performance of conventional methods when integrated with them, and consequently achieved the best records, surpassing even the existing hyperbolic metric learning technique, in almost all settings.