# Robust Reinforcement Learning via Adversarial training with Langevin Dynamics

arXiv: https://arxiv.org/abs/2002.06063 · HTML: https://mlsanity.com/paper/2002.06063/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2020.

## Built on by (most cited first, 30 of 45)

- [Maximum Entropy RL (Provably) Solves Some Robust RL Problems](https://mlsanity.com/paper/2103.06257/) (2021)
- [RORL: Robust Offline Reinforcement Learning via Conservative Smoothing](https://mlsanity.com/paper/2206.02829/) (2022)
- [Policy Smoothing for Provably Robust Reinforcement Learning](https://mlsanity.com/paper/2106.11420/) (2021)
- [Probabilistically Robust Learning: Balancing Average- and Worst-case Performance](https://mlsanity.com/paper/2202.01136/) (2022)
- [Robust Inverse Reinforcement Learning under Transition Dynamics Mismatch](https://mlsanity.com/paper/2007.01174/) (2020)
- [Robust Predictable Control](https://mlsanity.com/paper/2109.03214/) (2021)
- [Robust Reinforcement Learning as a Stackelberg Game via Adaptively-Regularized Adversarial Training](https://mlsanity.com/paper/2202.09514/) (2022)
- [No-Regret Learning in Games with Noisy Feedback: Faster Rates and Adaptivity via Learning Rate Separation](https://mlsanity.com/paper/2206.06015/) (2022)
- [Identifiability and generalizability from multiple experts in Inverse Reinforcement Learning](https://mlsanity.com/paper/2209.10974/) (2022)
- [Combining Pessimism with Optimism for Robust and Efficient Model-Based Deep Reinforcement Learning](https://mlsanity.com/paper/2103.10369/) (2021)
- [Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advances](https://mlsanity.com/paper/2508.10316/) (2025)
- [Max-Min Off-Policy Actor-Critic Method Focusing on Worst-Case Robustness to Model Misspecification](https://mlsanity.com/paper/2211.03413/) (2022)
- [Robust Adversarial Reinforcement Learning via Bounded Rationality Curricula](https://mlsanity.com/paper/2311.01642/) (2023)
- [Robust Learning from Observation with Model Misspecification](https://mlsanity.com/paper/2202.06003/) (2022)
- [MICRO: Model-Based Offline Reinforcement Learning with a Conservative Bellman Operator](https://mlsanity.com/paper/2312.03991/) (2023)
- [Robust Reinforcement Learning in Continuous Control Tasks with Uncertainty Set Regularization](https://mlsanity.com/paper/2207.02016/) (2022)
- [ReRoGCRL: Representation-based Robustness in Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2312.07392/) (2023)
- [Learning a Robust Multiagent Driving Policy for Traffic Congestion Reduction](https://mlsanity.com/paper/2112.03759/) (2021)
- [Adversarially Robust Decision Transformer](https://mlsanity.com/paper/2407.18414/) (2024)
- [ADV-0: Closed-Loop Min-Max Adversarial Training for Long-Tail Robustness in Autonomous Driving](https://mlsanity.com/paper/2603.15221/) (2026)
- [ACReL: Adversarial Conditional value-at-risk Reinforcement Learning](https://mlsanity.com/paper/2109.09470/) (2021)
- [Convergence of GANs Training: A Game and Stochastic Control Methodology](https://mlsanity.com/paper/2112.00222/) (2021)
- [Review of Metrics to Measure the Stability, Robustness and Resilience of Reinforcement Learning](https://mlsanity.com/paper/2203.12048/) (2022)
- [Robust Reinforcement Learning through Efficient Adversarial Herding](https://mlsanity.com/paper/2306.07408/) (2023)
- [Context-aware Diversity Enhancement for Neural Multi-Objective Combinatorial Optimization](https://mlsanity.com/paper/2405.08604/) (2024)
- [On Minimizing Adversarial Counterfactual Error in Adversarial RL](https://mlsanity.com/paper/2406.04724/) (2024)
- [RRLS : Robust Reinforcement Learning Suite](https://mlsanity.com/paper/2406.08406/) (2024)
- [Solving robust MDPs as a sequence of static RL problems](https://mlsanity.com/paper/2410.06212/) (2024)
- [Provable Robustness for Streaming Models with a Sliding Window](https://mlsanity.com/paper/2303.16308/) (2023)
- [Where2Start: Leveraging initial States for Robust and Sample-Efficient Reinforcement Learning](https://mlsanity.com/paper/2311.15089/) (2023)

## Abstract (arXiv)

We introduce a sampling perspective to tackle the challenging task of training robust Reinforcement Learning (RL) agents. Leveraging the powerful Stochastic Gradient Langevin Dynamics, we present a novel, scalable two-player RL algorithm, which is a sampling variant of the two-player policy gradient method. Our algorithm consistently outperforms existing baselines, in terms of generalization across different training and testing conditions, on several MuJoCo environments. Our experiments also show that, even for objective functions that entirely ignore potential environmental shifts, our sampling approach remains highly robust in comparison to standard RL algorithms.