mlsanityData to 14 Oct 2026

Improved Stochastic Optimization of LogSumExp

arXiv 2509.24894 ↗First posted 29 Sep 2025math.OC cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

The LogSumExp function, dual to the Kullback-Leibler (KL) divergence, plays a central role in many important optimization problems, including entropy-regularized optimal transport (OT) and distributionally robust optimization (DRO). In practice, when the number of exponential terms inside the logarithm is large or infinite, optimization becomes challenging since computing the gradient requires differentiating every term. We propose a novel convexity- and smoothness-preserving approximation to LogSumExp that can be efficiently optimized using stochastic gradient methods. This approximation is rooted in a sound modification of the KL divergence in the dual, resulting in a new f-divergence called the Safe KL divergence. Our experiments and theoretical analysis of the LogSumExp-based stochastic optimization, arising in DRO and continuous OT, demonstrate the advantages of our approach over existing baselines.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 32 of its 66 references that are in the map. First posted on arXiv 29 Sep 2025.

Builds on

32 of 66 references.

References · 32

  1. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  2. 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 449
  3. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 337
  4. 2015Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulationscited by 291
  5. 2016Stochastic Optimization for Large-scale Optimal Transportcited by 189
  6. 2020Large-Scale Methods for Distributionally Robust Optimizationcited by 179
  7. 2015Optimal Entropy-Transport problems and a new Hellinger-Kantorovich distance between positive measurescited by 139
  8. 2022Neural Optimal Transportcited by 126
  9. 2017Large-Scale Optimal Transport and Mapping Estimationcited by 123
  10. 2021The Bayesian Learning Rulecited by 99
  11. 2021Score-based Generative Neural Networks for Large-Scale Optimal Transportcited by 60
  12. 2022Entropic Neural Optimal Transport via Diffusion Processescited by 50
20 more
  1. 2018On the Fenchel Duality between Strong Convexity and Lipschitz Continuous Gradientcited by 43
  2. 2022Provable Stochastic Optimization for Global Contrastive Learning: Small Batch Does Not Harm Performancecited by 43
  3. 2020An Online Method for A Class of Distributionally Robust Optimization with Non-Convex Objectivescited by 42
  4. 2018Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Modelscited by 38
  5. 2021Sinkhorn Distributionally Robust Optimizationcited by 38
  6. 2022Neural Optimal Transport with General Cost Functionalscited by 36
  7. 2020Label Noise Types and Their Effects on Deep Learningcited by 29
  8. 2022Finite-Sum Coupled Compositional Stochastic Optimization: Theory and Applicationscited by 27
  9. 2023Energy-guided Entropic Neural Optimal Transportcited by 26
  10. 2020Statistical Learning with Conditional Value at Riskcited by 19
  11. 2013On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoodscited by 16
  12. 2016Guaranteed bounds on the Kullback-Leibler divergence of univariate mixtures using piecewise log-sum-exp inequalitiescited by 16
  13. 2017A New Unbiased and Efficient Class of LSH-Based Samplers and Estimators for Partition Function Computation in Log-Linear Modelscited by 16
  14. 2022Stochastic Constrained DRO with a Complexity Independent of Sample Sizecited by 16
  15. 2023Outlier-Robust Wasserstein DROcited by 16
  16. 2016One-vs-Each Approximation to Softmax for Scalable Estimation of Probabilitiescited by 14
  17. 2020SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Modelscited by 14
  18. 2020(f,Gamma)-Divergences: Interpolating between f-Divergences and Integral Probability Metricscited by 10
  19. 2023Demonstration-Regularized RLcited by 9
  20. 2023LSEMINK: A Modified Newton-Krylov Method for Log-Sum-Exp Minimizationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Hyperspherical Forward-Forward with Prototypical Representationscited by 3
  2. 2026A Geometry-Aware Efficient Algorithm for Compositional Entropic Risk Minimizationcited by 1
  3. 2026PERO: Efficient Robust Post-Training Foundation Models for Encrypted Traffic Classificationcited by 0
  4. 2026Scalable Cox Regression via Grouped Risk Sets and Sharper LogSumExp Ratescited by 0