# Tempered Calculus for ML: Application to Hyperbolic Model Embedding

arXiv: https://arxiv.org/abs/2402.04163 · HTML: https://mlsanity.com/paper/2402.04163/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 59 references that are in the map.
- First posted on arXiv 6 Feb 2024.

## Builds on (most cited first, 18 of 18)

- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Hyperbolic Neural Networks](https://mlsanity.com/paper/1805.09112/) (2018)
- [Representation Tradeoffs for Hyperbolic Embeddings](https://mlsanity.com/paper/1804.03329/) (2018)
- [Hyperbolic Entailment Cones for Learning Hierarchical Embeddings](https://mlsanity.com/paper/1804.01882/) (2018)
- [From Trees to Continuous Embeddings and Back: Hyperbolic Hierarchical Clustering](https://mlsanity.com/paper/2010.00402/) (2020)
- [HoroPCA: Hyperbolic Dimensionality Reduction via Horospherical Projections](https://mlsanity.com/paper/2106.03306/) (2021)
- [The Numerical Stability of Hyperbolic Representation Learning](https://mlsanity.com/paper/2211.00181/) (2022)
- [Tree! I am no Tree! I am a Low Dimensional Hyperbolic Embedding](https://mlsanity.com/paper/2005.03847/) (2020)
- [Hyperbolic Representation Learning: Revisiting and Advancing](https://mlsanity.com/paper/2306.09118/) (2023)
- [Hyperbolic Random Forests](https://mlsanity.com/paper/2308.13279/) (2023)
- [Boosting with Tempered Exponential Measures](https://mlsanity.com/paper/2306.05487/) (2023)
- [Boosted Density Estimation Remastered](https://mlsanity.com/paper/1803.08178/) (2018)
- [Supervised Learning: No Loss No Cry](https://mlsanity.com/paper/2002.03555/) (2020)
- [Fast hyperboloid decision tree algorithms](https://mlsanity.com/paper/2310.13841/) (2023)
- [Clustering above Exponential Families with Tempered Exponential Measures](https://mlsanity.com/paper/2211.02765/) (2022)
- [Optimal Transport with Tempered Exponential Measures](https://mlsanity.com/paper/2309.04015/) (2023)
- [The Tempered Hilbert Simplex Distance and Its Application To Non-linear Embeddings of TEMs](https://mlsanity.com/paper/2311.13459/) (2023)

## Abstract (arXiv)

Most mathematical distortions used in ML are fundamentally integral in nature: f-divergences, Bregman divergences, (regularized) optimal transport distances, integral probability metrics, geodesic distances, etc. In this paper, we unveil a grounded theory and tools which can help improve these distortions to better cope with ML requirements. We start with a generalization of Riemann integration that also encapsulates functions that are not strictly additive but are, more generally, t-additive, as in nonextensive statistical mechanics. Notably, this recovers Volterra's product integral as a special case. We then generalize the Fundamental Theorem of calculus using an extension of the (Euclidean) derivative. This, along with a series of more specific Theorems, serves as a basis for results showing how one can specifically design, alter, or change fundamental properties of distortion measures in a simple way, with a special emphasis on geometric- and ML-related properties that are the metricity, hyperbolicity, and encoding. We show how to apply it to a problem that has recently gained traction in ML: hyperbolic embeddings with a "cheap" and accurate encoding along the hyperbolic vs Euclidean scale. We unveil a new application for which the Poincar'e disk model has very appealing features, and our theory comes in handy: textitmodel embeddings for boosted combinations of decision trees, trained using the log-loss (trees) and logistic loss (combinations).