Schedule Based Temporal Difference Algorithms
Not on the map. Why?
Abstract (arXiv)
Learning the value function of a given policy from data samples is an important problem in Reinforcement Learning. TD(λ) is a popular class of algorithms to solve this problem. However, the weights assigned to different n-step returns in TD(λ), controlled by the parameter λ, decrease exponentially with increasing n. In this paper, we present a λ-schedule procedure that generalizes the TD(λ) algorithm to the case when the parameter λ could vary with time-step. This allows flexibility in weight assignment, i.e., the user can specify the weights assigned to different n-step returns by choosing a sequence λtt ≥ 1. Based on this procedure, we propose an on-policy algorithm - TD(λ)-schedule, and two off-policy algorithms - GTD(λ)-schedule and TDC(λ)-schedule, respectively. We provide proofs of almost sure convergence for all three algorithms under a general Markov noise framework.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 5 of its 27 references that are in the map. First posted on arXiv 23 Nov 2021.
Builds on
5 of 27 references.
References · 5
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
- 2015Stability of Stochastic Approximations with `Controlled Markov' Noise and Temporal Difference Learningcited by 9
- 2018Online Off-policy Predictioncited by 9
- 2015Two Timescale Stochastic Approximation with Controlled Markov noise and Off-policy temporal difference learningcited by 8
- 2018An Incremental Off-policy Search in a Model-free Markov Decision Process Using a Single Sample Pathcited by 1
Built on by
Not cited yet.