# Attention-based Ensemble for Deep Metric Learning

arXiv: https://arxiv.org/abs/1804.00382 · HTML: https://mlsanity.com/paper/1804.00382/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2018.

## Built on by (most cited first, 30 of 56)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Hyperbolic Vision Transformers: Combining Improvements in Metric Learning](https://mlsanity.com/paper/2203.10833/) (2022)
- [Training Vision Transformers for Image Retrieval](https://mlsanity.com/paper/2102.05644/) (2021)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation](https://mlsanity.com/paper/2101.05544/) (2021)
- [Learning Intra-Batch Connections for Deep Metric Learning](https://mlsanity.com/paper/2102.07753/) (2021)
- [Non-isotropy Regularization for Proxy-based Deep Metric Learning](https://mlsanity.com/paper/2203.08547/) (2022)
- [Deep Relational Metric Learning](https://mlsanity.com/paper/2108.10026/) (2021)
- [Embedding Transfer with Label Relaxation for Improved Metric Learning](https://mlsanity.com/paper/2103.14908/) (2021)
- [Learning with Memory-based Virtual Classes for Deep Metric Learning](https://mlsanity.com/paper/2103.16940/) (2021)
- [Integrating Language Guidance into Vision-based Deep Metric Learning](https://mlsanity.com/paper/2203.08543/) (2022)
- [Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning](https://mlsanity.com/paper/2103.15454/) (2021)
- [Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learning](https://mlsanity.com/paper/2107.09562/) (2021)
- [Orthogonal Ensemble Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/2105.10827/) (2021)
- [Self-Taught Metric Learning without Labels](https://mlsanity.com/paper/2205.01903/) (2022)
- [Attributable Visual Similarity Learning](https://mlsanity.com/paper/2203.14932/) (2022)
- [Understanding Hyperbolic Metric Learning through Hard Negative Sampling](https://mlsanity.com/paper/2404.15523/) (2024)
- [Adaptive Hierarchical Similarity Metric Learning with Noisy Labels](https://mlsanity.com/paper/2111.00006/) (2021)
- [Contrastive Bayesian Analysis for Deep Metric Learning](https://mlsanity.com/paper/2210.04402/) (2022)
- [Semi-Supervised Learning via Weight-aware Distillation under Class Distribution Mismatch](https://mlsanity.com/paper/2308.11874/) (2023)
- [LoOp: Looking for Optimal Hard Negative Embeddings for Deep Metric Learning](https://mlsanity.com/paper/2108.09335/) (2021)
- [Introspective Deep Metric Learning](https://mlsanity.com/paper/2309.09982/) (2023)
- [Multi-level Distance Regularization for Deep Metric Learning](https://mlsanity.com/paper/2102.04223/) (2021)
- [The Group Loss++: A deeper look into group loss for deep metric learning](https://mlsanity.com/paper/2204.01509/) (2022)
- [V2M: Visual 2-Dimensional Mamba for Image Representation Learning](https://mlsanity.com/paper/2410.10382/) (2024)
- [Group Contrastive Self-Supervised Learning on Graphs](https://mlsanity.com/paper/2107.09787/) (2021)
- [Improving Deep Metric Learning by Divide and Conquer](https://mlsanity.com/paper/2109.04003/) (2021)
- [DAS: Densely-Anchored Sampling for Deep Metric Learning](https://mlsanity.com/paper/2208.00119/) (2022)
- [Multi-Head Deep Metric Learning Using Global and Local Representations](https://mlsanity.com/paper/2112.14327/) (2021)
- [Rethinking Deep Contrastive Learning with Embedding Memory](https://mlsanity.com/paper/2103.14003/) (2021)

## Abstract (arXiv)

Deep metric learning aims to learn an embedding function, modeled as deep neural network. This embedding function usually puts semantically similar images close while dissimilar images far from each other in the learned embedding space. Recently, ensemble has been applied to deep metric learning to yield state-of-the-art results. As one important aspect of ensemble, the learners should be diverse in their feature embeddings. To this end, we propose an attention-based ensemble, which uses multiple attention masks, so that each learner can attend to different parts of the object. We also propose a divergence loss, which encourages diversity among the learners. The proposed method is applied to the standard benchmarks of deep metric learning and experimental results show that it outperforms the state-of-the-art methods by a significant margin on image retrieval tasks.