mlsanityData to 7 Oct 2026

Maximum a Posteriori Policy Optimisation

arXiv 1806.06920 ↗First posted 14 Jun 2018cs.LG cs.AI cs.ITcited by 318

Not on the map. Why?

Cited by 45 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce a new algorithm for reinforcement learning called Maximum aposteriori Policy Optimisation (MPO) based on coordinate ascent on a relative entropy objective. We show that several existing methods can directly be related to our derivation. We develop two off-policy algorithms and demonstrate that they are competitive with the state-of-the-art in deep reinforcement learning. In particular, for continuous control, our method outperforms existing methods with respect to sample efficiency, premature convergence and robustness to hyperparameter settings while achieving similar or better final performance.

Key facts. Cited by 318 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 4; 2020: 5; 2021: 58; 2022: 65; 2023: 44; 2024: 45; 2025: 46; 2026: 51.

318 papers.

Sort:

By area:

  1. 2023Mastering Diverse Domains through World Modelscited by 1,193
  2. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  3. 2022A Generalist Agentcited by 802
  4. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  5. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  6. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  7. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  8. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 213
  9. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
  10. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  11. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  12. 2024AgentGym: Evolving Large Language Model-based Agents across Diverse Environmentscited by 120
38 more
  1. 2023Extreme Q-Learning: MaxEnt RL without Entropycited by 113
  2. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 111
  3. 2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 108
  4. 2023RoboCat: A Self-Improving Generalist Agent for Robotic Manipulationcited by 108
  5. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperationcited by 103
  6. 2023Confronting Reward Model Overoptimization with Constrained RLHFcited by 96
  7. 2020Human-centric Dialog Training via Offline Reinforcement Learningcited by 92
  8. 2021AndroidEnv: A Reinforcement Learning Platform for Androidcited by 90
  9. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  10. 2021Reward is enough for convex MDPscited by 82
  11. 2021Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learningcited by 78
  12. 2023Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedbackcited by 71
  13. 2024Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbonecited by 71
  14. 2021Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapescited by 70
  15. 2022Constrained Variational Policy Optimization for Safe Reinforcement Learningcited by 69
  16. 2022COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimationcited by 69
  17. 2021Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learningcited by 65
  18. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learningcited by 64
  19. 2020RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learningcited by 57
  20. 2024Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agentscited by 56
  21. 2024A Dense Reward View on Aligning Text-to-Image Diffusion with Preferencecited by 55
  22. 2021Concave Utility Reinforcement Learning: the Mean-Field Game Viewpointcited by 54
  23. 2021Muesli: Combining Improvements in Policy Optimizationcited by 53
  24. 2022Retrieval-Augmented Reinforcement Learningcited by 53
  25. 2024Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewardingcited by 53
  26. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 50
  27. 2023Dual RL: Unification and New Methods for Reinforcement and Imitation Learningcited by 48
  28. 2021Bootstrapped Meta-Learningcited by 45
  29. 2024Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedbackcited by 45
  30. 2026RL Token: Bootstrapping Online RL with Vision-Language-Action Modelscited by 43
  31. 2021Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent Statescited by 42
  32. 2024WARP: On the Benefits of Weight Averaged Rewarded Policiescited by 42
  33. 2025Beyond Reward Hacking: Causal Rewards for Large Language Model Alignmentcited by 42
  34. 2022NeRF2Real: Sim2real Transfer of Vision-guided Bipedal Motion Skills using Neural Radiance Fieldscited by 41
  35. 2021Greedification Operators for Policy Optimization: Investigating Forward and Reverse KL Divergencescited by 40
  36. 2022DARA: Dynamics-Aware Reward Augmentation in Offline Reinforcement Learningcited by 40
  37. 2021Baby Intuitions Benchmark (BIB): Discerning the goals, preferences, and actions of otherscited by 39
  38. 2025Supervised Fine Tuning on Curated Data is Reinforcement Learning (and can be improved)cited by 37

50 most cited of 318 All 318 (JSON)