mlsanityData to 7 Oct 2026

Robust Adversarial Reinforcement Learning

arXiv 1703.02702 ↗First posted 8 Mar 2017cs.LG cs.AI cs.MAcited by 439

Not on the map. Why?

Cited by 27 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep neural networks coupled with fast simulation and improved computation have led to recent successes in the field of reinforcement learning (RL). However, most current RL-based approaches fail to generalize since: (a) the gap between simulation and real world is so large that policy-learning approaches fail to transfer; (b) even if policy learning is done in real world, the data scarcity leads to failed generalization from training to test scenarios (e.g., due to different friction or object masses). Inspired from H-infinity control methods, we note that both modeling errors and differences in training and test scenarios can be viewed as extra forces/disturbances in the system. This paper proposes the idea of robust adversarial reinforcement learning (RARL), where we train an agent to operate in the presence of a destabilizing adversary that applies disturbance forces to the system. The jointly trained adversary is reinforced -- that is, it learns an optimal destabilization policy. We formulate the policy learning as a zero-sum, minimax objective function. Extensive experiments in multiple environments (InvertedPendulum, HalfCheetah, Swimmer, Hopper and Walker2d) conclusively demonstrate that our method (a) improves training stability; (b) is robust to differences in training/test conditions; and c) outperform the baseline even in the absence of the adversary.

Key facts. Cited by 439 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 76; 2022: 77; 2023: 79; 2024: 62; 2025: 78; 2026: 65.

439 papers.

Sort:

By area:

  1. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  2. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  3. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  4. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
  5. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  6. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 133
  7. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
  8. 2022RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learningcited by 115
  9. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
  10. 2021Online Robust Reinforcement Learning with Model Uncertaintycited by 95
  11. 2022Robust Reinforcement Learning using Offline Datacited by 95
  12. 2022RORL: Robust Offline Reinforcement Learning via Conservative Smoothingcited by 83
38 more
  1. 2021Sample Complexity of Robust Reinforcement Learning with a Generative Modelcited by 82
  2. 2023Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulationcited by 78
  3. 2022Policy Gradient Method For Robust Reinforcement Learningcited by 67
  4. 2021Robot Learning from Randomized Simulations: A Reviewcited by 56
  5. 2024R.A.C.E.: Robust Adversarial Concept Erasure for Secure Text-to-Image Diffusion Modelcited by 56
  6. 2023How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligencecited by 55
  7. 2021Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RLcited by 54
  8. 2023Double Pessimism is Provably Efficient for Distributionally Robust Offline Reinforcement Learning: Generic Algorithm and Robust Partial Coveragecited by 51
  9. 2021Twice regularized MDPs and the equivalence between robustness and regularizationcited by 46
  10. 2022Sound Adversarial Audio-Visual Navigationcited by 46
  11. 2023Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstractioncited by 45
  12. 2023Improved Sample Complexity Bounds for Distributionally Robust Reinforcement Learningcited by 44
  13. 2021safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Roboticscited by 42
  14. 2022Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learningcited by 42
  15. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 41
  16. 2022Efficient Risk-Averse Reinforcement Learningcited by 41
  17. 2021Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challengescited by 40
  18. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 40
  19. 2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
  20. 2021Adversarial Reinforcement Learning for Procedural Content Generationcited by 36
  21. 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatchcited by 33
  22. 2021Context-Aware Safe Reinforcement Learning for Non-Stationary Environmentscited by 33
  23. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
  24. 2024Reward-Robust RLHF in LLMscited by 32
  25. 2021Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimizationcited by 31
  26. 2022Distributionally Robust Offline Reinforcement Learning with Linear Function Approximationcited by 31
  27. 2021Robust Policy Gradient against Strong Data Corruptioncited by 30
  28. 2021Unsupervised Visual Attention and Invariance for Reinforcement Learningcited by 30
  29. 2021Towards Distraction-Robust Active Visual Trackingcited by 30
  30. 2021Provably Efficient Black-Box Action Poisoning Attacks Against Reinforcement Learningcited by 29
  31. 2023Policy Gradient for Rectangular Robust Markov Decision Processescited by 29
  32. 2024No Regrets: Investigating and Improving Regret Approximations for Curriculum Discoverycited by 29
  33. 2021Risk-Averse Bayes-Adaptive Reinforcement Learningcited by 28
  34. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  35. 2022Robust Reinforcement Learning as a Stackelberg Game via Adaptively-Regularized Adversarial Trainingcited by 26
  36. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  37. 2021Evaluating the Robustness of Collaborative Agentscited by 25
  38. 2023Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithmscited by 25

50 most cited of 439 All 439 (JSON)