mlsanityData to 7 Oct 2026

Three Dogmas of Reinforcement Learning

arXiv 2407.10583 ↗First posted 15 Jul 2024cs.AI cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Modern reinforcement learning has been conditioned by at least three dogmas. The first is the environment spotlight, which refers to our tendency to focus on modeling environments rather than agents. The second is our treatment of learning as finding the solution to a task, rather than adaptation. The third is the reward hypothesis, which states that all goals and purposes can be well thought of as maximization of a reward signal. These three dogmas shape much of what we think of as the science of reinforcement learning. While each of the dogmas have played an important role in developing the field, it is time we bring them to the surface and reflect on whether they belong as basic ingredients of our scientific paradigm. In order to realize the potential of reinforcement learning as a canonical frame for researching intelligent agents, we suggest that it is time we shed dogmas one and two entirely, and embrace a nuanced approach to the third.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 42 of its 126 references that are in the map. First posted on arXiv 15 Jul 2024.

Builds on

42 of 126 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  7. 2016Cooperative Inverse Reinforcement Learningcited by 298
  8. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
  9. 2019Batch Policy Learning under Constraintscited by 242
  10. 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 183
  11. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
  12. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 155
30 more
  1. 2013Risk-sensitive Reinforcement Learningcited by 129
  2. 2016Reinforcement Learning With Temporal Logic Rewardscited by 114
  3. 2023A Definition of Continual Reinforcement Learningcited by 98
  4. 2021Reward is enough for convex MDPscited by 82
  5. 2024Robust agents learn causal world modelscited by 81
  6. 2021On the Expressivity of Markov Rewardcited by 71
  7. 2017Environment-Independent Task Specifications via GLTLcited by 67
  8. 2021Reinforcement Learning, Bit by Bitcited by 59
  9. 2013Sample Complexity of Multi-task Reinforcement Learningcited by 58
  10. 2019Hyperbolic Discounting and Learning over Multiple Horizonscited by 58
  11. 2021Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent Statescited by 42
  12. 2023Continual Learning as Computationally Constrained Reinforcement Learningcited by 35
  13. 2022Settling the Reward Hypothesiscited by 33
  14. 2019Rethinking the Discount Factor in Reinforcement Learning: A Decision Theoretic Approachcited by 32
  15. 2023Near-Minimax-Optimal Risk-Sensitive Reinforcement Learning with CVaRcited by 32
  16. 2020Action and Perception as Divergence Minimizationcited by 31
  17. 2023MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learningcited by 30
  18. 2022Challenging Common Assumptions in Convex Reinforcement Learningcited by 29
  19. 2022Discovering Agentscited by 28
  20. 2021Multi-Agent Reinforcement Learning with Temporal Logic Specificationscited by 27
  21. 2015Information-Theoretic Bounded Rationalitycited by 21
  22. 2019On Value Functions and the Agent-Environment Boundarycited by 19
  23. 2023Prediction and Control in Continual Reinforcement Learningcited by 19
  24. 2022The Quest for a Common Model of the Intelligent Decision Makercited by 16
  25. 2016Reinforcement Learning in Rich-Observation MDPs using Spectral Methodscited by 13
  26. 2016Nonparametric General Reinforcement Learningcited by 10
  27. 2019A Strongly Asymptotically Optimal Agent in General Environmentscited by 9
  28. 2022Utility Theory for Sequential Decision Makingcited by 8
  29. 2018The Barbados 2018 List of Open Issues in Continual Learningcited by 7
  30. 2023On the Convergence of Bounded Agentscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 6; 2026: 6.

14 papers.

Sort:

By area:

  1. 2025Rethinking the Foundations for Continual Reinforcement Learningcited by 15
  2. 2024Boundless Socratic Learning with Language Gamescited by 12
  3. 2025Plasticity as the Mirror of Empowermentcited by 11
  4. 2024Possible Principles for Aligned Structure Learning Agentscited by 3
  5. 2025The Formalism-Implementation Gap in Reinforcement Learning Researchcited by 3
  6. 2025Agent-centric learning: from external reward maximization to internal knowledge curationcited by 2
  7. 2026Objective Decoupling in Social Reinforcement Learning: Recovering Ground Truth from Sycophantic Majoritiescited by 2
  8. 2025The World Is Bigger! A Computationally-Embedded Perspective on the Big World Hypothesiscited by 1
  9. 2026Position: Deployed Reinforcement Learning should be Continualcited by 1
  10. 2025Illuminating the Three Dogmas of Reinforcement Learning under Evolutionary Lightcited by 0
  11. 2026Darwin Mobile Agent: A Roadmap for Self-Evolutioncited by 0
  12. 2026Compositional Behavioral Semantics for State Abstraction in Reinforcement Learningcited by 0
2 more
  1. 2026Efficient Exploration Is Enoughcited by 0
  2. 2026From Static Policies to Adaptive Priors in Offline Reinforcement Learningcited by 0