# Spectral, Probabilistic, and Deep Metric Learning: Tutorial and Survey

arXiv: https://arxiv.org/abs/2201.09267 · HTML: https://mlsanity.com/paper/2201.09267/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 147 references that are in the map.
- First posted on arXiv 23 Jan 2022.

## Builds on (most cited first, 30 of 54)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples](https://mlsanity.com/paper/1903.03096/) (2019)
- [Sampling Matters in Deep Embedding Learning](https://mlsanity.com/paper/1706.07567/) (2017)
- [Unsupervised Embedding Learning via Invariant and Spreading Instance Feature](https://mlsanity.com/paper/1904.03436/) (2019)
- [No Fuss Distance Metric Learning using Proxies](https://mlsanity.com/paper/1703.07464/) (2017)
- [SoftTriple Loss: Deep Metric Learning Without Triplet Sampling](https://mlsanity.com/paper/1909.05235/) (2019)
- [A Riemannian Network for SPD Matrix Learning](https://mlsanity.com/paper/1608.04233/) (2016)
- [Cross-Batch Memory for Embedding Learning](https://mlsanity.com/paper/1912.06798/) (2019)
- [Deep Metric Learning with Angular Loss](https://mlsanity.com/paper/1708.01682/) (2017)
- [ProxyNCA++: Revisiting and Revitalizing Proxy Neighborhood Component Analysis](https://mlsanity.com/paper/2004.01113/) (2020)
- [A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses](https://mlsanity.com/paper/2003.08983/) (2020)
- [Metric Learning for Adversarial Robustness](https://mlsanity.com/paper/1909.00900/) (2019)
- [Unsupervised Meta-Learning For Few-Shot Image Classification](https://mlsanity.com/paper/1811.11819/) (2018)
- [Improved Embeddings with Easy Positive Triplet Mining](https://mlsanity.com/paper/1904.04370/) (2019)
- [A Survey on Metric Learning for Feature Vectors and Structured Data](https://mlsanity.com/paper/1306.6709/) (2013)
- [PyTorch Metric Learning](https://mlsanity.com/paper/2008.09164/) (2020)
- [Eigenvalue and Generalized Eigenvalue Problems: Tutorial](https://mlsanity.com/paper/1903.11240/) (2019)
- [Reproducing Kernel Hilbert Space, Mercer's Theorem, Eigenfunctions, Nystr"om Method, and Use of Kernels in Machine Learning: Tutorial and Survey](https://mlsanity.com/paper/2106.08443/) (2021)
- [Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Long-Form Document Matching](https://mlsanity.com/paper/2004.12297/) (2020)
- [Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimising Global Loss Functions](https://mlsanity.com/paper/1512.09272/) (2015)
- [Generalized Contrastive Optimization of Siamese Networks for Place Recognition](https://mlsanity.com/paper/2103.06638/) (2021)
- [Deep Feature Learning with Relative Distance Comparison for Person Re-identification](https://mlsanity.com/paper/1512.03622/) (2015)
- [Geometric Mean Metric Learning](https://mlsanity.com/paper/1607.05002/) (2016)

## Built on by (most cited first, 10 of 10)

- [TaskMet: Task-Driven Metric Learning for Model Learning](https://mlsanity.com/paper/2312.05250/) (2023)
- [Pulling back symmetric Riemannian geometry for data analysis](https://mlsanity.com/paper/2403.06612/) (2024)
- [Pullback Flow Matching on Data Manifolds](https://mlsanity.com/paper/2410.04543/) (2024)
- [Causal Fair Metric: Bridging Causality, Individual Fairness, and Adversarial Robustness](https://mlsanity.com/paper/2310.19391/) (2023)
- [Affective Manifolds: Modeling Machine's Mind to Like, Dislike, Enjoy, Suffer, Worry, Fear, and Feel Like A Human](https://mlsanity.com/paper/2208.13386/) (2022)
- [Clone-Robust Weights in Metric Spaces: Handling Redundancy Bias for Benchmark Aggregation](https://mlsanity.com/paper/2502.03576/) (2025)
- [Unsupervised Ground Metric Learning](https://mlsanity.com/paper/2507.13094/) (2025)
- [Representer Theorems for Metric and Preference Learning: Geometric Insights and Algorithms](https://mlsanity.com/paper/2304.03720/) (2023)
- [Improved Aircraft Environmental Impact Segmentation via Metric Learning](https://mlsanity.com/paper/2306.13830/) (2023)
- [Foundations of Riemannian Geometry for Riemannian Optimization: A Monograph with Detailed Derivations](https://mlsanity.com/paper/2605.02279/) (2026)

## Abstract (arXiv)

This is a tutorial and survey paper on metric learning. Algorithms are divided into spectral, probabilistic, and deep metric learning. We first start with the definition of distance metric, Mahalanobis distance, and generalized Mahalanobis distance. In spectral methods, we start with methods using scatters of data, including the first spectral metric learning, relevant methods to Fisher discriminant analysis, Relevant Component Analysis (RCA), Discriminant Component Analysis (DCA), and the Fisher-HSIC method. Then, large-margin metric learning, imbalanced metric learning, locally linear metric adaptation, and adversarial metric learning are covered. We also explain several kernel spectral methods for metric learning in the feature space. We also introduce geometric metric learning methods on the Riemannian manifolds. In probabilistic methods, we start with collapsing classes in both input and feature spaces and then explain the neighborhood component analysis methods, Bayesian metric learning, information theoretic methods, and empirical risk minimization in metric learning. In deep learning methods, we first introduce reconstruction autoencoders and supervised loss functions for metric learning. Then, Siamese networks and its various loss functions, triplet mining, and triplet sampling are explained. Deep discriminant analysis methods, based on Fisher discriminant analysis, are also reviewed. Finally, we introduce multi-modal deep metric learning, geometric metric learning by neural networks, and few-shot metric learning.