Online Reinforcement Learning for Periodic MDP
Not on the map. Why?
Abstract (arXiv)
We study learning in periodic Markov Decision Process(MDP), a special type of non-stationary MDP where both the state transition probabilities and reward functions vary periodically, under the average reward maximization setting. We formulate the problem as a stationary MDP by augmenting the state space with the period index, and propose a periodic upper confidence bound reinforcement learning-2 (PUCRL2) algorithm. We show that the regret of PUCRL2 varies linearly with the period and as sub-linear with the horizon length. Numerical results demonstrate the efficacy of PUCRL2.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 21 references that are in the map. First posted on arXiv 25 Jul 2022.
Builds on
8 of 21 references.
References · 8
- 2021Non-stationary Reinforcement Learning without Prior Knowledge: An Optimal Black-box Approachcited by 98
- 2020Reinforcement Learning for Non-Stationary Markov Decision Processes: The Blessing of (More) Optimismcited by 76
- 2020Dynamic Regret of Policy Optimization in Non-stationary Environmentscited by 33
- 2019Variational Regret Bounds for Reinforcement Learningcited by 32
- 2020A Kernel-Based Approach to Non-Stationary Reinforcement Learning in Metric Spacescited by 32
- 2018A Sliding-Window Algorithm for Markov Decision Processes with Arbitrarily Changing Rewards and Transitionscited by 30
- 2020Tightening Exploration in Upper Confidence Reinforcement Learningcited by 25
- 2020Nonstationary Reinforcement Learning with Linear Function Approximationcited by 23
Built on by
Not cited yet.