# Reinforcement learning of rare diffusive dynamics

arXiv: https://arxiv.org/abs/2105.04321 · HTML: https://mlsanity.com/paper/2105.04321/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 138 references that are in the map.
- First posted on arXiv 10 May 2021.

## Builds on (most cited first, 15 of 15)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)
- [A Theory of Regularized Markov Decision Processes](https://mlsanity.com/paper/1901.11275/) (2019)
- [A unified view of entropy-regularized Markov decision processes](https://mlsanity.com/paper/1705.07798/) (2017)
- [Solving for high dimensional committor functions using artificial neural networks](https://mlsanity.com/paper/1802.10275/) (2018)
- [Computing Committor Functions for the Study of Rare Events Using Deep Learning](https://mlsanity.com/paper/1906.06285/) (2019)
- [Adaptive importance sampling for control and inference](https://mlsanity.com/paper/1505.01874/) (2015)
- [A reinforcement learning approach to rare trajectory sampling](https://mlsanity.com/paper/2005.12890/) (2020)
- [Policy Gradient based Quantum Approximate Optimization Algorithm](https://mlsanity.com/paper/2002.01068/) (2020)
- [Path Integral Control and State Dependent Feedback](https://mlsanity.com/paper/1406.4026/) (2014)
- [Deep Reinforcement Learning of Transition States](https://mlsanity.com/paper/2011.06700/) (2020)
- [Evolutionary reinforcement learning of dynamical large deviations](https://mlsanity.com/paper/1909.00835/) (2019)
- [Stochastic Optimal Control as Non-equilibrium Statistical Mechanics: Calculus of Variations over Density and Current](https://mlsanity.com/paper/1306.6572/) (2013)

## Built on by (most cited first, 15 of 15)

- [Stochastic Optimal Control for Collective Variable Free Sampling of Molecular Transition Paths](https://mlsanity.com/paper/2207.02149/) (2022)
- [Doob's Lagrangian: A Sample-Efficient Variational Approach to Transition Path Sampling](https://mlsanity.com/paper/2410.07974/) (2024)
- [Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inference](https://mlsanity.com/paper/2508.12511/) (2025)
- [Action-Minimization Meets Generative Modeling: Efficient Transition Path Sampling with the Onsager-Machlup Functional](https://mlsanity.com/paper/2504.18506/) (2025)
- [Transition Path Sampling with Improved Off-Policy Training of Diffusion Path Samplers](https://mlsanity.com/paper/2405.19961/) (2024)
- [Rare Event Analysis of Large Language Models](https://mlsanity.com/paper/2602.06791/) (2026)
- [Generative methods for sampling transition paths in molecular dynamics](https://mlsanity.com/paper/2205.02818/) (2022)
- [Entropy Regularized Reinforcement Learning Using Large Deviation Theory](https://mlsanity.com/paper/2106.03931/) (2021)
- [Combining Reinforcement Learning and Tensor Networks, with an Application to Dynamical Large Deviations](https://mlsanity.com/paper/2209.14089/) (2022)
- [Generalized Flow Matching for Transition Dynamics Modeling](https://mlsanity.com/paper/2410.15128/) (2024)
- [Generative artificial intelligence for computational chemistry: a roadmap to predicting emergent phenomena](https://mlsanity.com/paper/2409.03118/) (2024)
- [Probing reaction channels via reinforcement learning](https://mlsanity.com/paper/2305.17531/) (2023)
- [Entangled Schr"odinger Bridge Matching](https://mlsanity.com/paper/2511.07406/) (2025)
- [Thermodynamics of Reinforcement Learning Curricula](https://mlsanity.com/paper/2603.12324/) (2026)
- [Langevin Flow Maps: Efficient Molecular Dynamics and Transition Path Sampling](https://mlsanity.com/paper/2610.05998/) (2026)

## Abstract (arXiv)

We present a method to probe rare molecular dynamics trajectories directly using reinforcement learning. We consider trajectories that are conditioned to transition between regions of configuration space in finite time, like those relevant in the study of reactive events, as well as trajectories exhibiting rare fluctuations of time-integrated quantities in the long time limit, like those relevant in the calculation of large deviation functions. In both cases, reinforcement learning techniques are used to optimize an added force that minimizes the Kullback-Leibler divergence between the conditioned trajectory ensemble and a driven one. Under the optimized added force, the system evolves the rare fluctuation as a typical one, affording a variational estimate of its likelihood in the original trajectory ensemble. Low variance gradients employing value functions are proposed to increase the convergence of the optimal force. The method we develop employing these gradients leads to efficient and accurate estimates of both the optimal force and the likelihood of the rare event for a variety of model systems.