mlsanityData to 7 Oct 2026

Trust Region Policy Optimization

arXiv 1502.05477 ↗First posted 19 Feb 2015cs.LGcited by 2,503

Not on the map. Why?

Cited by 279 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We describe an iterative procedure for optimizing policies, with guaranteed monotonic improvement. By making several approximations to the theoretically-justified procedure, we develop a practical algorithm, called Trust Region Policy Optimization (TRPO). This algorithm is similar to natural policy gradient methods and is effective for optimizing large nonlinear policies such as neural networks. Our experiments demonstrate its robust performance on a wide variety of tasks: learning simulated robotic swimming, hopping, and walking gaits; and playing Atari games using images of the screen as input. Despite its approximations that deviate from the theory, TRPO tends to give monotonic improvement, with little tuning of hyperparameters.

Key facts. Cited by 2,503 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Feb 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2017: 1; 2018: 19; 2019: 29; 2020: 32; 2021: 426; 2022: 373; 2023: 384; 2024: 349; 2025: 442; 2026: 448.

2,503 papers.

Sort:

By area:

  1. 2017Deep reinforcement learning from human preferencescited by 3,846
  2. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  4. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  5. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  6. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  7. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  8. 2018Behavioral Cloning from Observationcited by 482
  9. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 463
  10. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  11. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  12. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
38 more
  1. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
  2. 2019Causal Confusion in Imitation Learningcited by 286
  3. 2026Reinforcement Learning via Self-Distillationcited by 285
  4. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  5. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  6. 2025Learning to Reason under Off-Policy Guidancecited by 266
  7. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 244
  8. 2021Mastering Atari Games with Limited Datacited by 235
  9. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  10. 2023Secrets of RLHF in Large Language Models Part I: PPOcited by 226
  11. 2021Likelihood Training of Schr"odinger Bridge using Forward-Backward SDEs Theorycited by 216
  12. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 213
  13. 2022Dual Diffusion Implicit Bridges for Image-to-Image Translationcited by 212
  14. 2026Self-Distillation Enables Continual Learningcited by 212
  15. 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learningcited by 210
  16. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 203
  17. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  18. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 194
  19. 2022Online Decision Transformercited by 194
  20. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 180
  21. 2025Training Deep Learning Models with Norm-Constrained LMOscited by 177
  22. 2024Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferencescited by 176
  23. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  24. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  25. 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 168
  26. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  27. 2019Goal-conditioned Imitation Learningcited by 159
  28. 2022Hybrid RL: Using Both Offline and Online Data Can Make RL Efficientcited by 158
  29. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
  30. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  31. 2023Efficient Diffusion Policies for Offline Reinforcement Learningcited by 151
  32. 2021Offline RL Without Off-Policy Evaluationcited by 150
  33. 2024A Minimaximalist Approach to Reinforcement Learning from Human Feedbackcited by 141
  34. 2025SimpleVLA-RL: Scaling VLA Training via Reinforcement Learningcited by 141
  35. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  36. 2024Token-level Direct Preference Optimizationcited by 135
  37. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 133
  38. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 124

50 most cited of 2,503 All 2,503 (JSON)