mlsanityData to 7 Oct 2026

State-free Reinforcement Learning

arXiv 2409.18439 ↗First posted 27 Sep 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

In this work, we study the textitstate-free RL problem, where the algorithm does not have the states information before interacting with the environment. Specifically, denote the reachable state set by S^Pi := s|maxπ∈ PiqP, π(s)>0 , we design an algorithm which requires no information on the state space S while having a regret that is completely independent of S and only depend on S^Pi. We view this as a concrete first step towards textitparameter-free RL, with the goal of designing RL algorithms that require no hyper-parameter tuning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 39 of its 49 references that are in the map. First posted on arXiv 27 Sep 2024.

Builds on

39 of 49 references.

References · 39

  1. 2017Minimax Regret Bounds for Reinforcement Learningcited by 500
  2. 2018Is Q-learning Provably Efficient?cited by 485
  3. 2019Tighter Problem-Dependent Regret Bounds in Reinforcement Learning without Domain Knowledge using Value Function Boundscited by 194
  4. 2017Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learningcited by 175
  5. 2016Coin Betting and Parameter-Free Online Learningcited by 131
  6. 2020Almost Optimal Model-Free Reinforcement Learning via Reference-Advantage Decompositioncited by 127
  7. 2018Black-Box Reductions for Parameter-free Online Learning in Banach Spacescited by 124
  8. 2023Learning-Rate-Free Learning by D-Adaptationcited by 113
  9. 2019Non-Asymptotic Gap-Dependent Regret Bounds for Tabular MDPscited by 104
  10. 2020Episodic Reinforcement Learning in Finite MDPs: Minimax Lower Bounds Revisitedcited by 104
  11. 2020Is Reinforcement Learning More Difficult Than Bandits? A Near-optimal Algorithm Escaping the Curse of Horizoncited by 99
  12. 2023DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedulecited by 85
27 more
  1. 2016Scale-Free Online Learningcited by 83
  2. 2019Provably Efficient Q-Learning with Low Switching Costcited by 77
  3. 2020Model Selection in Contextual Stochastic Bandit Problemscited by 71
  4. 2013Follow the Leader If You Can, Hedge If You Mustcited by 68
  5. 2018Efficient Bias-Span-Constrained Exploration-Exploitation in Reinforcement Learningcited by 65
  6. 2019Model selection for contextual banditscited by 60
  7. 2021Policy Optimization in Adversarial MDPs: Improved Exploration via Dilated Bonusescited by 51
  8. 2021Breaking the Sample Complexity Barrier to Regret-Optimal Model-Free Reinforcement Learningcited by 51
  9. 2022Making SGD Parameter-Freecited by 50
  10. 2020Bias no more: high-probability data-dependent regret bounds for adversarial bandits and MDPscited by 49
  11. 2021Fine-Grained Gap-Dependent Bounds for Tabular MDPs via Adaptive Multi-Step Bootstrapcited by 48
  12. 2021UCB Momentum Q-learning: Correcting the bias without forgettingcited by 45
  13. 2023Settling the Sample Complexity of Online Reinforcement Learningcited by 44
  14. 2021Beyond No Regret: Instance-Dependent PAC Reinforcement Learningcited by 41
  15. 2023Variance-Dependent Regret Bounds for Linear Bandits and Reinforcement Learning: Adaptivity and Computational Efficiencycited by 39
  16. 2017Parameter-free online learning via model selectioncited by 34
  17. 2020Planning in Markov Decision Processes with Gap-Dependent Sample Complexitycited by 34
  18. 2021Beyond Value-Function Gaps: Improved Instance-Dependent Regret Bounds for Episodic Reinforcement Learningcited by 33
  19. 2022PDE-Based Optimal Strategy for Unconstrained Online Learningcited by 27
  20. 2023Mechanic: A Learning Rate Tunercited by 20
  21. 2023Sharp Variance-Dependent Bounds in Reinforcement Learning: Best of Both Worlds in Stochastic and Deterministic Environmentscited by 17
  22. 2022Follow-the-Perturbed-Leader for Adversarial Markov Decision Processes with Bandit Feedbackcited by 16
  23. 2021A Fully Problem-Dependent Regret Lower Bound for Finite-Horizon MDPscited by 15
  24. 2024Tuning-Free Stochastic Optimizationcited by 13
  25. 2020Comparator-adaptive Convex Banditscited by 11
  26. 2023Improved Algorithms for Adversarial Bandits with Unbounded Lossescited by 3
  27. 2024Scale-free Adversarial Reinforcement Learningcited by 2

Built on by

Not cited yet.