# Neural Bregman Divergences for Distance Learning

arXiv: https://arxiv.org/abs/2206.04763 · HTML: https://mlsanity.com/paper/2206.04763/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 40 references that are in the map.
- First posted on arXiv 9 Jun 2022.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [Input Convex Neural Networks](https://mlsanity.com/paper/1609.07152/) (2016)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [Kernel Operations on the GPU, with Autodiff, without Memory Overflows](https://mlsanity.com/paper/2004.11127/) (2020)
- [Optimal Control Via Neural Networks: A Convex Approach](https://mlsanity.com/paper/1805.11835/) (2018)
- [Deep Metric Learning with Spherical Embedding](https://mlsanity.com/paper/2011.02785/) (2020)
- [An Inductive Bias for Distances: Neural Nets that Respect the Triangle Inequality](https://mlsanity.com/paper/2002.05825/) (2020)
- [Local Similarity-Aware Deep Feature Embedding](https://mlsanity.com/paper/1610.08904/) (2016)
- [Deep Divergence Learning](https://mlsanity.com/paper/2005.02612/) (2020)
- [Faster Algorithms for Learning Convex Functions](https://mlsanity.com/paper/2111.01348/) (2021)
- [BrePartition: Optimized High-Dimensional kNN Search with Bregman Distances](https://mlsanity.com/paper/2006.00227/) (2020)
- [Decomposing reverse-mode automatic differentiation](https://mlsanity.com/paper/2105.09469/) (2021)
- [A directed isoperimetric inequality with application to Bregman near neighbor lower bounds](https://mlsanity.com/paper/1404.1191/) (2014)

## Built on by (most cited first, 3 of 3)

- [Configurable Mirror Descent: Towards a Unification of Decision Making](https://mlsanity.com/paper/2405.11746/) (2024)
- [Learning Empirical Bregman Divergence for Uncertain Distance Representation](https://mlsanity.com/paper/2304.07689/) (2023)
- [Role-Aware Neural Convex Divergence Heads for Asymmetric Representation Learning](https://mlsanity.com/paper/2607.01762/) (2026)

## Abstract (arXiv)

Many metric learning tasks, such as triplet learning, nearest neighbor retrieval, and visualization, are treated primarily as embedding tasks where the ultimate metric is some variant of the Euclidean distance (e.g., cosine or Mahalanobis), and the algorithm must learn to embed points into the pre-chosen space. The study of non-Euclidean geometries is often not explored, which we believe is due to a lack of tools for learning non-Euclidean measures of distance. Recent work has shown that Bregman divergences can be learned from data, opening a promising approach to learning asymmetric distances. We propose a new approach to learning arbitrary Bergman divergences in a differentiable manner via input convex neural networks and show that it overcomes significant limitations of previous works. We also demonstrate that our method more faithfully learns divergences over a set of both new and previously studied tasks, including asymmetric regression, ranking, and clustering. Our tests further extend to known asymmetric, but non-Bregman tasks, where our method still performs competitively despite misspecification, showing the general utility of our approach for asymmetric learning.