Diffusion at Absolute Zero: Langevin Sampling Using Successive Moreau Envelopes [conference paper]

In this article we propose a novel method for sampling from Gibbs distributions of the form π(x)proptoexp(-U(x)) with a potential U(x). In particular, inspired by diffusion models we propose to consider a sequence (πtk)k of approximations of the target density, for which πtk≈ π for k small and, on the other hand, πtk exhibits favorable properties for sampling for k large. This sequence is obtained by replacing parts of the potential U by its Moreau envelopes. Sampling is performed in an Annealed Langevin type procedure, that is, sequentially sampling from πtk for decreasing k, effectively guiding the samples from a simple starting density to the more complex target. In addition to a theoretical analysis we show experimental results supporting the efficacy of the method in terms of increased convergence speed and applicability to multi-modal densities π.