# Atlas Based Representation and Metric Learning on Manifolds

arXiv: https://arxiv.org/abs/2106.07062 · HTML: https://mlsanity.com/paper/2106.07062/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 13 Jun 2021.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Wasserstein Auto-Encoders](https://mlsanity.com/paper/1711.01558/) (2017)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [No Fuss Distance Metric Learning using Proxies](https://mlsanity.com/paper/1703.07464/) (2017)
- [ProxyNCA++: Revisiting and Revitalizing Proxy Neighborhood Component Analysis](https://mlsanity.com/paper/2004.01113/) (2020)
- [Smart Mining for Deep Metric Learning](https://mlsanity.com/paper/1704.01285/) (2017)
- [Improved Embeddings with Easy Positive Triplet Mining](https://mlsanity.com/paper/1904.04370/) (2019)
- [Attention-based Ensemble for Deep Metric Learning](https://mlsanity.com/paper/1804.00382/) (2018)
- [Learning Deep Embeddings with Histogram Loss](https://mlsanity.com/paper/1611.00822/) (2016)
- [A Theoretically Sound Upper Bound on the Triplet Loss for Improving the Efficiency of Deep Distance Metric Learning](https://mlsanity.com/paper/1904.08720/) (2019)
- [Deep Randomized Ensembles for Metric Learning](https://mlsanity.com/paper/1808.04469/) (2018)
- [Adversarial Autoencoders with Constant-Curvature Latent Manifolds](https://mlsanity.com/paper/1812.04314/) (2018)
- [LDLE: Low Distortion Local Eigenmaps](https://mlsanity.com/paper/2101.11055/) (2021)
- [Autoencoding topology](https://mlsanity.com/paper/1803.00156/) (2018)

## Built on by (most cited first, 3 of 3)

- [Manifold Learning by Mixture Models of VAEs for Inverse Problems](https://mlsanity.com/paper/2303.15244/) (2023)
- [State Representation Learning Using an Unbalanced Atlas](https://mlsanity.com/paper/2305.10267/) (2023)
- [Maximum Manifold Capacity Representations in State Representation Learning](https://mlsanity.com/paper/2405.13848/) (2024)

## Abstract (arXiv)

We explore the use of a topological manifold, represented as a collection of charts, as the target space of neural network based representation learning tasks. This is achieved by a simple adjustment to the output of an encoder's network architecture plus the addition of a maximal mean discrepancy (MMD) based loss function for regularization. Most algorithms in representation and metric learning are easily adaptable to our framework and we demonstrate its effectiveness by adjusting SimCLR (for representation learning) and standard triplet loss training (for metric learning) to have manifold encoding spaces. Our experiments show that we obtain a substantial performance boost over the baseline for low dimensional encodings. In the case of triplet training, we also find, independent of the manifold setup, that the MMD loss alone (i.e. keeping a flat, euclidean target space but using an MMD loss to regularize it) increases performance over the baseline in the typical, high-dimensional Euclidean target spaces. Code for reproducing experiments is provided at https://github.com/ekorman/neurve .