mlsanityData to 7 Oct 2026

Exploration Unbound

arXiv 2407.12178 ↗First posted 16 Jul 2024cs.LG cs.AI stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

A sequential decision-making agent balances between exploring to gain new knowledge about an environment and exploiting current knowledge to maximize immediate reward. For environments studied in the traditional literature, optimal decisions gravitate over time toward exploitation as the agent accumulates sufficient knowledge and the benefits of further exploration vanish. What if, however, the environment offers an unlimited amount of useful knowledge and there is large benefit to further exploration no matter how much the agent has learned? We offer a simple, quintessential example of such a complex environment. In this environment, rewards are unbounded and an agent can always increase the rate at which rewards accumulate by exploring to learn more. Consequently, an optimal agent forever maintains a propensity to explore.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 59 references that are in the map. First posted on arXiv 16 Jul 2024.

Builds on

21 of 59 references.

References · 21

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2020Learning to summarize from human feedbackcited by 2,562
  3. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  4. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  5. 2017A Tutorial on Thompson Samplingcited by 420
  6. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
  7. 2023A Definition of Continual Reinforcement Learningcited by 98
  8. 2021AndroidEnv: A Reinforcement Learning Platform for Androidcited by 90
  9. 2021Reinforcement Learning, Bit by Bitcited by 59
  10. 2024Efficient Exploration for LLMscited by 44
  11. 2015Simple regret for infinitely many armed banditscited by 39
  12. 2018Satisficing in Time-Sensitive Bandit Learningcited by 38
9 more
  1. 2023Continual Learning as Computationally Constrained Reinforcement Learningcited by 35
  2. 2021Deciding What to Learn: A Rate-Distortion Approachcited by 23
  3. 2022Deciding What to Model: Value-Equivalent Sampling for Reinforcement Learningcited by 19
  4. 2020On Regret with Multiple Best Armscited by 16
  5. 2021Bandits with many optimal armscited by 16
  6. 2018Pure Exploration in Infinitely-Armed Bandit Models with Fixed-Confidencecited by 15
  7. 2021The Value of Information When Deciding What to Learncited by 15
  8. 2021From Finite to Countable-Armed Banditscited by 8
  9. 2023Bayesian Reinforcement Learning with Limited Cognitive Loadcited by 4

Built on by

Not cited yet.