# Mirror Descent and Novel Exponentiated Gradient Algorithms Using Trace-Form Entropies and Deformed Logarithms

arXiv: https://arxiv.org/abs/2503.08748 · HTML: https://mlsanity.com/paper/2503.08748/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 69 references that are in the map.
- First posted on arXiv 11 Mar 2025.

## Builds on (most cited first, 9 of 9)

- [The Information Geometry of Mirror Descent](https://mlsanity.com/paper/1310.7780/) (2013)
- [Exponentiated Gradient Meets Gradient Descent](https://mlsanity.com/paper/1902.01903/) (2019)
- [Mirrorless Mirror Descent: A Natural Derivation of Mirror Descent](https://mlsanity.com/paper/2004.01025/) (2020)
- [Boosting with Tempered Exponential Measures](https://mlsanity.com/paper/2306.05487/) (2023)
- [Generalized Exponentiated Gradient Algorithms and Their Application to On-Line Portfolio Selection](https://mlsanity.com/paper/2406.00655/) (2024)
- [Conformal Mirror Descent with Logarithmic Divergences](https://mlsanity.com/paper/2209.02938/) (2022)
- [Optimal Transport with Tempered Exponential Measures](https://mlsanity.com/paper/2309.04015/) (2023)
- [Generalized Euler Logarithm and its Applications in Machine Learning: Natural Gradient, Backpropagation, Generalized EG, Mirror Descent and OLPS](https://mlsanity.com/paper/2502.17500/) (2025)
- [The Generalized Bregman Distance](https://mlsanity.com/paper/1909.08206/) (2019)

## Built on by (most cited first, 3 of 3)

- [The alpha-beta divergence for real and complex data](https://mlsanity.com/paper/2508.03272/) (2025)
- [Policy Regret for Embedding Model Routing: Contextual Bandits with Low-Rank Experts](https://mlsanity.com/paper/2606.14929/) (2026)
- [Group Entropies and Mirror Duality: A Class of Flexible Mirror Descent Updates for Machine Learning](https://mlsanity.com/paper/2603.08651/) (2026)

## Abstract (arXiv)

This paper introduces a broad class of Mirror Descent (MD) and Generalized Exponentiated Gradient (GEG) algorithms derived from trace-form entropies defined via deformed logarithms. Leveraging these generalized entropies yields MD & GEG algorithms with improved convergence behavior, robustness to vanishing and exploding gradients, and inherent adaptability to non-Euclidean geometries through mirror maps. We establish deep connections between these methods and Amari's natural gradient, revealing a unified geometric foundation for additive, multiplicative, and natural gradient updates. Focusing on the Tsallis, Kaniadakis, Sharma--Taneja--Mittal, and Kaniadakis--Lissia--Scarfone entropy families, we show that each entropy induces a distinct Riemannian metric on the parameter space, leading to GEG algorithms that preserve the natural statistical geometry. The tunable parameters of deformed logarithms enable adaptive geometric selection, providing enhanced robustness and convergence over classical Euclidean optimization. Overall, our framework unifies key first-order MD optimization methods under a single information-geometric perspective based on generalized Bregman divergences, where the choice of entropy determines the underlying metric and dual geometric structure.