# Active Importance Sampling for Variational Objectives Dominated by Rare Events: Consequences for Optimization and Generalization

arXiv: https://arxiv.org/abs/2008.06334 · HTML: https://mlsanity.com/paper/2008.06334/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Aug 2020.

## Built on by (most cited first, 5 of 5)

- [Interpolating between BSDEs and PINNs: deep learning for elliptic and parabolic boundary value problems](https://mlsanity.com/paper/2112.03749/) (2021)
- [A Finite Expression Method for Solving High-Dimensional Committor Problems](https://mlsanity.com/paper/2306.12268/) (2023)
- [Generative methods for sampling transition paths in molecular dynamics](https://mlsanity.com/paper/2205.02818/) (2022)
- [Is the neural tangent kernel of PINNs deep learning general partial differential equations always convergent ?](https://mlsanity.com/paper/2412.06158/) (2024)
- [Overview of Physics-Informed Machine Learning Inversion of Geophysical Data](https://mlsanity.com/paper/2310.08109/) (2023)

## Abstract (arXiv)

Deep neural networks, when optimized with sufficient data, provide accurate representations of high-dimensional functions; in contrast, function approximation techniques that have predominated in scientific computing do not scale well with dimensionality. As a result, many high-dimensional sampling and approximation problems once thought intractable are being revisited through the lens of machine learning. While the promise of unparalleled accuracy may suggest a renaissance for applications that require parameterizing representations of complex systems, in many applications gathering sufficient data to develop such a representation remains a significant challenge. Here we introduce an approach that combines rare events sampling techniques with neural network optimization to optimize objective functions that are dominated by rare events. We show that importance sampling reduces the asymptotic variance of the solution to a learning problem, suggesting benefits for generalization. We study our algorithm in the context of learning dynamical transition pathways between two states of a system, a problem with applications in statistical physics and implications in machine learning theory. Our numerical experiments demonstrate that we can successfully learn even with the compounding difficulties of high-dimension and rare data.