# Unsupervised Deep Metric Learning with Transformed Attention Consistency and Contrastive Clustering Loss

arXiv: https://arxiv.org/abs/2008.04378 · HTML: https://mlsanity.com/paper/2008.04378/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Aug 2020.

## Built on by (most cited first, 18 of 18)

- [PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clustering](https://mlsanity.com/paper/2103.17070/) (2021)
- [Unicom: Universal and Compact Representation Learning for Image Retrieval](https://mlsanity.com/paper/2304.05884/) (2023)
- [Integrating Language Guidance into Vision-based Deep Metric Learning](https://mlsanity.com/paper/2203.08543/) (2022)
- [Self-Taught Metric Learning without Labels](https://mlsanity.com/paper/2205.01903/) (2022)
- [Piecewise-Linear Manifolds for Deep Metric Learning](https://mlsanity.com/paper/2403.14977/) (2024)
- [A Survey of Historical Learning: Learning Models with Learning History](https://mlsanity.com/paper/2303.12992/) (2023)
- [SLIQ: Quantum Image Similarity Networks on Noisy Quantum Computers](https://mlsanity.com/paper/2309.15259/) (2023)
- [ATCON: Attention Consistency for Vision Models](https://mlsanity.com/paper/2210.09705/) (2022)
- [FlameFinder: Illuminating Obscured Fire through Smoke with Attentive Deep Metric Learning](https://mlsanity.com/paper/2404.06653/) (2024)
- [Amodal SAM: A Unified Amodal Segmentation Framework with Generalization](https://mlsanity.com/paper/2604.20748/) (2026)
- [DeepDiffusion: Unsupervised Learning of Retrieval-adapted Representations via Diffusion-based Ranking on Latent Feature Manifold](https://mlsanity.com/paper/2112.07082/) (2021)
- [Color Variants Identification in Fashion e-commerce via Contrastive Self-Supervised Representation Learning](https://mlsanity.com/paper/2104.08581/) (2021)
- [A Tale of Color Variants: Representation and Self-Supervised Learning in Fashion E-Commerce](https://mlsanity.com/paper/2112.02910/) (2021)
- [Hub-VAE: Unsupervised Hub-based Regularization of Variational Autoencoders](https://mlsanity.com/paper/2211.10469/) (2022)
- [Embracing Annotation Efficient Learning (AEL) for Digital Pathology and Natural Images](https://mlsanity.com/paper/2212.00470/) (2022)
- [Multi-view Action Recognition via Directed Gromov-Wasserstein Discrepancy](https://mlsanity.com/paper/2405.01337/) (2024)
- [AI-CNet3D: An Anatomically-Informed Cross-Attention Network with Multi-Task Consistency Fine-tuning for 3D Glaucoma Classification](https://mlsanity.com/paper/2510.00882/) (2025)
- [Concept Retrieval -- What and How?](https://mlsanity.com/paper/2510.07058/) (2025)

## Abstract (arXiv)

Existing approaches for unsupervised metric learning focus on exploring self-supervision information within the input image itself. We observe that, when analyzing images, human eyes often compare images against each other instead of examining images individually. In addition, they often pay attention to certain keypoints, image regions, or objects which are discriminative between image classes but highly consistent within classes. Even if the image is being transformed, the attention pattern will be consistent. Motivated by this observation, we develop a new approach to unsupervised deep metric learning where the network is learned based on self-supervision information across images instead of within one single image. To characterize the consistent pattern of human attention during image comparisons, we introduce the idea of transformed attention consistency. It assumes that visually similar images, even undergoing different image transforms, should share the same consistent visual attention map. This consistency leads to a pairwise self-supervision loss, allowing us to learn a Siamese deep neural network to encode and compare images against their transformed or matched pairs. To further enhance the inter-class discriminative power of the feature generated by this network, we adapt the concept of triplet loss from supervised metric learning to our unsupervised case and introduce the contrastive clustering loss. Our extensive experimental results on benchmark datasets demonstrate that our proposed method outperforms current state-of-the-art methods for unsupervised metric learning by a large margin.