mlsanityData to 7 Oct 2026

Bayesian Reinforcement Learning with Limited Cognitive Load

arXiv 2305.03263 ↗First posted 5 May 2023cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

All biological and artificial agents must learn and make decisions given limits on their ability to process information. As such, a general theory of adaptive behavior should be able to account for the complex interactions between an agent's learning history, decisions, and capacity constraints. Recent work in computer science has begun to clarify the principles that shape these dynamics by bridging ideas from reinforcement learning, Bayesian decision-making, and rate-distortion theory. This body of work provides an account of capacity-limited Bayesian reinforcement learning, a unifying normative framework for modeling the effect of processing constraints on learning and action selection. Here, we provide an accessible review of recent algorithms and theoretical results in this setting, paying special attention to how these ideas can be applied to studying questions in the cognitive and behavioral sciences.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 70 of its 222 references that are in the map. First posted on arXiv 5 May 2023.

Builds on

70 of 222 references.

References · 70

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2021Convex optimizationcited by 2,479
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  4. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  5. 2016Deep Exploration via Bootstrapped DQNcited by 518
  6. 2017Minimax Regret Bounds for Reinforcement Learningcited by 500
  7. 2018Is Q-learning Provably Efficient?cited by 485
  8. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
  9. 2017A Tutorial on Thompson Samplingcited by 420
  10. 2013(More) Efficient Reinforcement Learning via Posterior Samplingcited by 233
  11. 2020Model-Based Reinforcement Learning with Value-Targeted Regressioncited by 233
  12. 2021The Statistical Complexity of Interactive Decision Makingcited by 208
38 more
  1. 2019Tighter Problem-Dependent Regret Bounds in Reinforcement Learning without Domain Knowledge using Value Function Boundscited by 194
  2. 2014An Information-Theoretic Analysis of Thompson Samplingcited by 178
  3. 2017Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learningcited by 175
  4. 2014Learning to Optimize via Information-Directed Samplingcited by 168
  5. 2017Deep Exploration via Randomized Value Functionscited by 155
  6. 2015Taming the Noise in Reinforcement Learning via Soft Updatescited by 125
  7. 2017Value Prediction Networkcited by 117
  8. 2014Model-based Reinforcement Learning and the Eluder Dimensioncited by 116
  9. 2016Why is Posterior Sampling Better than Optimism for Reinforcement Learning?cited by 116
  10. 2014Generalization and Exploration via Randomized Value Functionscited by 112
  11. 2015Sample Complexity of Episodic Fixed-Horizon Reinforcement Learningcited by 105
  12. 2016The Predictron: End-To-End Learning and Planningcited by 92
  13. 2018Lipschitz Continuity in Model-based Reinforcement Learningcited by 90
  14. 2020The Value Equivalence Principle for Model-Based Reinforcement Learningcited by 88
  15. 2019InfoBot: Transfer and Exploration via the Information Bottleneckcited by 78
  16. 2014Algorithms for multi-armed bandit problemscited by 63
  17. 2021Reinforcement Learning, Bit by Bitcited by 59
  18. 2013Sample Complexity of Multi-task Reinforcement Learningcited by 58
  19. 2019Information asymmetry in KL-regularized RLcited by 53
  20. 2019Information-Theoretic Confidence Bounds for Reinforcement Learningcited by 51
  21. 2015Increasing the Action Gap: New Operators for Reinforcement Learningcited by 49
  22. 2020Goal-Aware Prediction: Learning to Model What Matterscited by 45
  23. 2021People construct simplified mental representations to plancited by 45
  24. 2013Thompson Sampling for Complex Bandit Problemscited by 43
  25. 2021Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent Statescited by 42
  26. 2019An Information-Theoretic Approach to Minimax Regret in Partial Monitoringcited by 41
  27. 2020Mirror Descent and the Information Ratiocited by 39
  28. 2018Satisficing in Time-Sensitive Bandit Learningcited by 38
  29. 2020Hypermodels for Explorationcited by 37
  30. 2021Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiationcited by 35
  31. 2021Proper Value Equivalencecited by 33
  32. 2022Value Gradient weighted Model-Based Reinforcement Learningcited by 31
  33. 2019Reinforcement Learning with Competitive Ensembles of Information-Constrained Primitivescited by 30
  34. 2020Making Sense of Reinforcement Learning and Probabilistic Inferencecited by 30
  35. 2021Generalized Kernel Thinningcited by 29
  36. 2013Prior-free and prior-dependent regret bounds for Thompson Samplingcited by 25
  37. 2022Regret Bounds for Information-Directed Reinforcement Learningcited by 24
  38. 2019Connections Between Mirror Descent, Thompson Sampling and the Information Ratiocited by 23

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 1.

4 papers.

  1. 2024Satisficing Exploration for Deep Reinforcement Learningcited by 4
  2. 2025The Alignment Bottleneckcited by 1
  3. 2023Balancing utility and cognitive cost in social representationcited by 0
  4. 2024Exploration Unboundcited by 0