mlsanityData to 7 Oct 2026

On Rate-Distortion Theory in Capacity-Limited Cognition & Reinforcement Learning

arXiv 2210.16877 ↗First posted 30 Oct 2022cs.LG cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Throughout the cognitive-science literature, there is widespread agreement that decision-making agents operating in the real world do so under limited information-processing capabilities and without access to unbounded cognitive or computational resources. Prior work has drawn inspiration from this fact and leveraged an information-theoretic model of such behaviors or policies as communication channels operating under a bounded rate constraint. Meanwhile, a parallel line of work also capitalizes on the same principles from rate-distortion theory to formalize capacity-limited decision making through the notion of a learning target, which facilitates Bayesian regret bounds for provably-efficient learning algorithms. In this paper, we aim to elucidate this latter perspective by presenting a brief survey of these information-theoretic models of capacity-limited decision making in biological and artificial agents.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 53 of its 137 references that are in the map. First posted on arXiv 30 Oct 2022.

Builds on

53 of 137 references.

References · 53

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2021Convex optimizationcited by 2,479
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  4. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  5. 2016Deep Exploration via Bootstrapped DQNcited by 518
  6. 2017Minimax Regret Bounds for Reinforcement Learningcited by 500
  7. 2018Is Q-learning Provably Efficient?cited by 485
  8. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
  9. 2013(More) Efficient Reinforcement Learning via Posterior Samplingcited by 233
  10. 2020Model-Based Reinforcement Learning with Value-Targeted Regressioncited by 233
  11. 2019Tighter Problem-Dependent Regret Bounds in Reinforcement Learning without Domain Knowledge using Value Function Boundscited by 194
  12. 2014An Information-Theoretic Analysis of Thompson Samplingcited by 178
38 more
  1. 2017Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learningcited by 175
  2. 2014Learning to Optimize via Information-Directed Samplingcited by 168
  3. 2017Deep Exploration via Randomized Value Functionscited by 155
  4. 2015Taming the Noise in Reinforcement Learning via Soft Updatescited by 125
  5. 2017Value Prediction Networkcited by 117
  6. 2014Model-based Reinforcement Learning and the Eluder Dimensioncited by 116
  7. 2016Why is Posterior Sampling Better than Optimism for Reinforcement Learning?cited by 116
  8. 2014Generalization and Exploration via Randomized Value Functionscited by 112
  9. 2015Sample Complexity of Episodic Fixed-Horizon Reinforcement Learningcited by 105
  10. 2017Building machines that adapt and compute like brainscited by 99
  11. 2016The Predictron: End-To-End Learning and Planningcited by 92
  12. 2018Lipschitz Continuity in Model-based Reinforcement Learningcited by 90
  13. 2020The Value Equivalence Principle for Model-Based Reinforcement Learningcited by 88
  14. 2021Reinforcement Learning, Bit by Bitcited by 59
  15. 2019Information asymmetry in KL-regularized RLcited by 53
  16. 2019Information-Theoretic Confidence Bounds for Reinforcement Learningcited by 51
  17. 2020Goal-Aware Prediction: Learning to Model What Matterscited by 45
  18. 2021People construct simplified mental representations to plancited by 45
  19. 2021Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent Statescited by 42
  20. 2018Satisficing in Time-Sensitive Bandit Learningcited by 38
  21. 2020Hypermodels for Explorationcited by 37
  22. 2021Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiationcited by 35
  23. 2021Proper Value Equivalencecited by 33
  24. 2022Value Gradient weighted Model-Based Reinforcement Learningcited by 31
  25. 2020Making Sense of Reinforcement Learning and Probabilistic Inferencecited by 30
  26. 2020Policy-Aware Model Learning for Policy Gradient Methodscited by 23
  27. 2021Deciding What to Learn: A Rate-Distortion Approachcited by 23
  28. 2021An Information-Theoretic Perspective on Credit Assignment in Reinforcement Learningcited by 20
  29. 2019Gradient-Aware Model-based Policy Searchcited by 19
  30. 2022Deciding What to Model: Value-Equivalent Sampling for Reinforcement Learningcited by 19
  31. 2019Towards a Quantum-Like Cognitive Architecture for Decision-Makingcited by 17
  32. 2019Exploiting Hierarchy for Learning and Transfer in KL-regularized RLcited by 15
  33. 2021The Value of Information When Deciding What to Learncited by 15
  34. 2014Bayesian Optimal Control of Smoothly Parameterized Systems: The Lazy Posterior Sampling Algorithmcited by 11
  35. 2017A Unified Bellman Equation for Causal Information and Value in Markov Decision Processescited by 11
  36. 2020Control-Aware Representations for Model-based Reinforcement Learningcited by 8
  37. 2015Rationally inattentive control of Markov processescited by 6
  38. 2020The Efficiency of Human Cognition Reflects Planned Information Processingcited by 6

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 2.

6 papers.

  1. 2024An information-theoretic model of shallow and deep language comprehensioncited by 6
  2. 2024GLIMPSE: Pragmatically Informative Multi-Document Summarization for Scholarly Reviewscited by 6
  3. 2026Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directionscited by 5
  4. 2024Satisficing Exploration for Deep Reinforcement Learningcited by 4
  5. 2025Illuminating the Three Dogmas of Reinforcement Learning under Evolutionary Lightcited by 0
  6. 2026Adaptive state-action abstractions via rate-distortioncited by 0