# Mean-Field Microcanonical Gradient Descent

arXiv: https://arxiv.org/abs/2403.08362 · HTML: https://mlsanity.com/paper/2403.08362/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 13 Mar 2024.

## Builds on (most cited first, 15 of 15)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Understanding Deep Convolutional Networks](https://mlsanity.com/paper/1601.04920/) (2016)
- [Sig-Wasserstein GANs for Time Series Generation](https://mlsanity.com/paper/2111.01207/) (2021)
- [Kymatio: Scattering Transforms in Python](https://mlsanity.com/paper/1812.11214/) (2018)
- [A Data-driven Market Simulator for Small Data Environments](https://mlsanity.com/paper/2006.14498/) (2020)
- [Scattering Networks for Hybrid Representation Learning](https://mlsanity.com/paper/1809.06367/) (2018)
- [Multiscale Sparse Microcanonical Models](https://mlsanity.com/paper/1801.02013/) (2018)
- [Maximum Entropy Models from Phase Harmonic Covariances](https://mlsanity.com/paper/1911.10017/) (2019)
- [Joint Time-Frequency Scattering](https://mlsanity.com/paper/1807.08869/) (2018)
- [Scattering Spectra Models for Physics](https://mlsanity.com/paper/2306.17210/) (2023)
- [Scale Dependencies and Self-Similar Models with Wavelet Scattering Spectra](https://mlsanity.com/paper/2204.10177/) (2022)
- [Time-Frequency Scattering Accurately Models Auditory Similarities Between Instrumental Playing Techniques](https://mlsanity.com/paper/2007.10926/) (2020)
- [Generalized Rectifier Wavelet Covariance Models For Texture Synthesis](https://mlsanity.com/paper/2203.07902/) (2022)

## Abstract (arXiv)

Microcanonical gradient descent is a sampling procedure for energy-based models allowing for efficient sampling of distributions in high dimension. It works by transporting samples from a high-entropy distribution, such as Gaussian white noise, to a low-energy region using gradient descent. We put this model in the framework of normalizing flows, showing how it can often overfit by losing an unnecessary amount of entropy in the descent. As a remedy, we propose a mean-field microcanonical gradient descent that samples several weakly coupled data points simultaneously, allowing for better control of the entropy loss while paying little in terms of likelihood fit. We study these models in the context of financial time series, illustrating the improvements on both synthetic and real data.