# A reinforcement learning approach to rare trajectory sampling

arXiv: https://arxiv.org/abs/2005.12890 · HTML: https://mlsanity.com/paper/2005.12890/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 May 2020.

## Built on by (most cited first, 22 of 22)

- [Doob's Lagrangian: A Sample-Efficient Variational Approach to Transition Path Sampling](https://mlsanity.com/paper/2410.07974/) (2024)
- [Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inference](https://mlsanity.com/paper/2508.12511/) (2025)
- [Action-Minimization Meets Generative Modeling: Efficient Transition Path Sampling with the Onsager-Machlup Functional](https://mlsanity.com/paper/2504.18506/) (2025)
- [Reinforcement learning of rare diffusive dynamics](https://mlsanity.com/paper/2105.04321/) (2021)
- [Rare Event Analysis of Large Language Models](https://mlsanity.com/paper/2602.06791/) (2026)
- [Generative methods for sampling transition paths in molecular dynamics](https://mlsanity.com/paper/2205.02818/) (2022)
- [Discrete generative diffusion models without stochastic differential equations: a tensor network approach](https://mlsanity.com/paper/2407.11133/) (2024)
- [Average-Reward Soft Actor-Critic](https://mlsanity.com/paper/2501.09080/) (2025)
- [Entropy Regularized Reinforcement Learning Using Large Deviation Theory](https://mlsanity.com/paper/2106.03931/) (2021)
- [Learning stochastic dynamics and predicting emergent behavior using transformers](https://mlsanity.com/paper/2202.08708/) (2022)
- [Combining Reinforcement Learning and Tensor Networks, with an Application to Dynamical Large Deviations](https://mlsanity.com/paper/2209.14089/) (2022)
- [Generalized Flow Matching for Transition Dynamics Modeling](https://mlsanity.com/paper/2410.15128/) (2024)
- [Pedestrian Trajectory Forecasting Using Deep Ensembles Under Sensing Uncertainty](https://mlsanity.com/paper/2305.16620/) (2023)
- [A Deep Reinforcement Learning Approach to Rare Event Estimation](https://mlsanity.com/paper/2211.12470/) (2022)
- [RL-PINNs: Reinforcement Learning-Driven Adaptive Sampling for Efficient Training of PINNs](https://mlsanity.com/paper/2504.12949/) (2025)
- [Probing reaction channels via reinforcement learning](https://mlsanity.com/paper/2305.17531/) (2023)
- [AtomComposer: Discovering Chemical Space from First Principles with Reinforcement Learning](https://mlsanity.com/paper/2605.28287/) (2026)
- [Learn Quasi-stationary Distributions of Finite State Markov Chain](https://mlsanity.com/paper/2111.11213/) (2021)
- [Augmenting Control over Exploration Space in Molecular Dynamics Simulators to Streamline De Novo Analysis through Generative Control Policies](https://mlsanity.com/paper/2306.14705/) (2023)
- [Mixing Artificial and Natural Intelligence: From Statistical Mechanics to AI and Back to Turbulence](https://mlsanity.com/paper/2403.17993/) (2024)
- [Nonequilibrium Dynamics of Simple Exclusion Processes Across Dimensions](https://mlsanity.com/paper/2608.25606/) (2026)
- [Langevin Flow Maps: Efficient Molecular Dynamics and Transition Path Sampling](https://mlsanity.com/paper/2610.05998/) (2026)

## Abstract (arXiv)

Very often when studying non-equilibrium systems one is interested in analysing dynamical behaviour that occurs with very low probability, so called rare events. In practice, since rare events are by definition atypical, they are often difficult to access in a statistically significant way. What are required are strategies to "make rare events typical" so that they can be generated on demand. Here we present such a general approach to adaptively construct a dynamics that efficiently samples atypical events. We do so by exploiting the methods of reinforcement learning (RL), which refers to the set of machine learning techniques aimed at finding the optimal behaviour to maximise a reward associated with the dynamics. We consider the general perspective of dynamical trajectory ensembles, whereby rare events are described in terms of ensemble reweighting. By minimising the distance between a reweighted ensemble and that of a suitably parametrised controlled dynamics we arrive at a set of methods similar to those of RL to numerically approximate the optimal dynamics that realises the rare behaviour of interest. As simple illustrations we consider in detail the problem of excursions of a random walker, for the case of rare events with a finite time horizon; and the problem of a studying current statistics of a particle hopping in a ring geometry, for the case of an infinite time horizon. We discuss natural extensions of the ideas presented here, including to continuous-time Markov systems, first passage time problems and non-Markovian dynamics.