# Probabilistic Gaussian Homotopy: A Probability-Space Continuation Framework for Nonconvex Optimization

arXiv: https://arxiv.org/abs/2603.13546 · HTML: https://mlsanity.com/paper/2603.13546/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 37 references that are in the map.
- First posted on arXiv 13 Mar 2026.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Personalized Federated Learning with Moreau Envelopes](https://mlsanity.com/paper/2006.08848/) (2020)
- [Entropy-SGD: Biasing Gradient Descent Into Wide Valleys](https://mlsanity.com/paper/1611.01838/) (2016)
- [Theoretical guarantees for approximate sampling from smooth and log-concave densities](https://mlsanity.com/paper/1412.7392/) (2014)
- [Single Loop Gaussian Homotopy Method for Non-convex Optimization](https://mlsanity.com/paper/2203.05717/) (2022)
- [A Hamilton-Jacobi-based Proximal Operator](https://mlsanity.com/paper/2211.12997/) (2022)
- [Laplace Meets Moreau: Smooth Approximation to Infimal Convolutions Using Laplace's Method](https://mlsanity.com/paper/2406.02003/) (2024)
- [Global Solutions to Nonconvex Problems by Evolution of Hamilton-Jacobi PDEs](https://mlsanity.com/paper/2202.11014/) (2022)
- [Connecting Hamilton--Jacobi partial differential equations with maximum a posteriori and posterior mean estimators for some non-convex priors](https://mlsanity.com/paper/2104.11285/) (2021)
- [A Monte Carlo Approach for Nonsmooth Convex Optimization via Proximal Splitting Algorithms](https://mlsanity.com/paper/2509.07914/) (2025)
- [Operator Splitting with Hamilton-Jacobi-based Proximals](https://mlsanity.com/paper/2601.22370/) (2026)

## Abstract (arXiv)

We introduce Probabilistic Gaussian Homotopy (PGH), a probability-space continuation framework for nonconvex optimization. Unlike classical Gaussian homotopy, which smooths the objective and uniformly averages gradients, PGH deforms the associated Boltzmann distribution and induces Boltzmann-weighted aggregation of perturbed gradients, which exponentially biases descent directions toward low-energy regions. We show that PGH corresponds to a log-sum-exp (soft-min) homotopy that smooths a nonconvex objective at scale λ>0 and recovers the original objective as λ→ 0, yielding a posterior-mean generalization of the Moreau envelope, and we derive a dynamical system governing minimizer evolution along an annealed homotopy path. This establishes a principled connection between Gaussian continuation, Bayesian denoising, and diffusion-style smoothing. We further propose Probabilistic Gaussian Homotopy Optimization (PGHO), a practical stochastic algorithm based on Monte Carlo gradient estimation, and demonstrate strong performance on high-dimensional nonconvex benchmarks and sparse recovery problems where classical gradient methods and objective-space smoothing frequently fail.