# Detecting and Mitigating Mode-Collapse for Flow-based Sampling of Lattice Field Theories

arXiv: https://arxiv.org/abs/2302.14082 · HTML: https://mlsanity.com/paper/2302.14082/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 31 references that are in the map.
- First posted on arXiv 27 Feb 2023.

## Builds on (most cited first, 5 of 5)

- [Targeted free energy estimation via learned mappings](https://mlsanity.com/paper/2002.04913/) (2020)
- [Aspects of scaling and scalability for flow-based sampling of lattice QCD](https://mlsanity.com/paper/2211.07541/) (2022)
- [Gradients should stay on Path: Better Estimators of the Reverse- and Forward KL Divergence for Normalizing Flows](https://mlsanity.com/paper/2207.08219/) (2022)
- [Machine Learning of Thermodynamic Observables in the Presence of Mode Collapse](https://mlsanity.com/paper/2111.11303/) (2021)
- [Machine Learning Trivializing Maps: A First Step Towards Understanding How Flow-Based Samplers Scale Up](https://mlsanity.com/paper/2112.15532/) (2021)

## Built on by (most cited first, 25 of 25)

- [Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimization](https://mlsanity.com/paper/2310.02679/) (2023)
- [Diffusion Models as Stochastic Quantization in Lattice Field Theory](https://mlsanity.com/paper/2309.17082/) (2023)
- [A theoretical perspective on mode collapse in variational inference](https://mlsanity.com/paper/2410.13300/) (2024)
- [Generative Diffusion Models for Lattice Field Theory](https://mlsanity.com/paper/2311.03578/) (2023)
- [Sampling the lattice Nambu-Goto string using Continuous Normalizing Flows](https://mlsanity.com/paper/2307.01107/) (2023)
- [On learning higher-order cumulants in diffusion models](https://mlsanity.com/paper/2410.21212/) (2024)
- [Physics-Conditioned Diffusion Models for Lattice Gauge Theory](https://mlsanity.com/paper/2502.05504/) (2025)
- [Numerical determination of the width and shape of the effective string using Stochastic Normalizing Flows](https://mlsanity.com/paper/2409.15937/) (2024)
- [Multilevel Generative Samplers for Investigating Critical Phenomena](https://mlsanity.com/paper/2503.08918/) (2025)
- [Mitigating mode collapse in normalizing flows by annealing with an adaptive schedule: Application to parameter estimation](https://mlsanity.com/paper/2505.03652/) (2025)
- [Fast and Unified Path Gradient Estimators for Normalizing Flows](https://mlsanity.com/paper/2403.15881/) (2024)
- [Performance of machine-learning-assisted Monte Carlo in sampling from simple statistical physics models](https://mlsanity.com/paper/2505.22598/) (2025)
- [Annealing in variational inference mitigates mode collapse: A theoretical study on Gaussian mixtures](https://mlsanity.com/paper/2602.12923/) (2026)
- [Hierarchical autoregressive neural networks in three-dimensional statistical system](https://mlsanity.com/paper/2503.08610/) (2025)
- [Path Gradients after Flow Matching](https://mlsanity.com/paper/2505.10139/) (2025)
- [SESaMo: Symmetry-Enforcing Stochastic Modulation for Normalizing Flows](https://mlsanity.com/paper/2505.19619/) (2025)
- [Multi-Lattice Sampling of Quantum Field Theories via Neural Operator-based Flows](https://mlsanity.com/paper/2401.00828/) (2024)
- [Simulating the Hubbard Model with Equivariant Normalizing Flows](https://mlsanity.com/paper/2501.07371/) (2025)
- [AGMA: Adaptive Gaussian Mixture Anchors for Prior-Guided Multimodal Human Trajectory Forecasting](https://mlsanity.com/paper/2602.04204/) (2026)
- [Stochastic normalizing flows for Effective String Theory](https://mlsanity.com/paper/2412.19109/) (2024)
- [Score-Based Density Estimation from Pairwise Comparisons](https://mlsanity.com/paper/2510.09146/) (2025)
- [Variational Autoregressive Networks with probability priors](https://mlsanity.com/paper/2605.16020/) (2026)
- [Correlation-Free Transition Path Sampling through Shooting Point Generation Guided by Committor Learning](https://mlsanity.com/paper/2609.20461/) (2026)
- [SIMANF: Sample Free Learning of Unnormalized Distributions via Simulated Annealing in Normalizing Flows](https://mlsanity.com/paper/2609.32279/) (2026)
- [FUND: Density Flow for Sampling Unnormalised Distributions](https://mlsanity.com/paper/2609.32296/) (2026)

## Abstract (arXiv)

We study the consequences of mode-collapse of normalizing flows in the context of lattice field theory. Normalizing flows allow for independent sampling. For this reason, it is hoped that they can avoid the tunneling problem of local-update MCMC algorithms for multi-modal distributions. In this work, we first point out that the tunneling problem is also present for normalizing flows but is shifted from the sampling to the training phase of the algorithm. Specifically, normalizing flows often suffer from mode-collapse for which the training process assigns vanishingly low probability mass to relevant modes of the physical distribution. This may result in a significant bias when the flow is used as a sampler in a Markov-Chain or with Importance Sampling. We propose a metric to quantify the degree of mode-collapse and derive a bound on the resulting bias. Furthermore, we propose various mitigation strategies in particular in the context of estimating thermodynamic observables, such as the free energy.