mlsanityData to 7 Oct 2026

Full Gradient Deep Reinforcement Learning for Average-Reward Criterion

arXiv 2304.03729 ↗First posted 7 Apr 2023eess.SY cs.LG cs.SYcited by 5

Not on the map. Why?

Abstract (arXiv)

We extend the provably convergent Full Gradient DQN algorithm for discounted reward Markov decision processes from Avrachenkov et al. (2021) to average reward problems. We experimentally compare widely used RVI Q-Learning with recently proposed Differential Q-Learning in the neural function approximation setting with Full Gradient DQN and DQN. We also extend this to learn Whittle indices for Markovian restless multi-armed bandits. We observe a better convergence rate of the proposed Full Gradient variant across different tasks.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 9 of its 30 references that are in the map. First posted on arXiv 7 Apr 2023.

Builds on

9 of 30 references.

References · 9

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 422
  3. 2020Learning and Planning in Average-Reward Markov Decision Processescited by 54
  4. 2020Whittle index based Q-learning for restless bandits with average rewardcited by 43
  5. 2021NeurWIN: Neural Whittle Index Network For Restless Bandits Via Deep RLcited by 27
  6. 2021Concentration of Contractive Stochastic Approximation and Reinforcement Learningcited by 23
  7. 2020Average-reward model-free reinforcement learning: a systematic review and literature mappingcited by 19
  8. 2016Deadline Scheduling as Restless Banditscited by 16
  9. 2021Full Gradient DQN Reinforcement Learning: A Provably Convergent Schemecited by 8

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 3; 2025: 1.

5 papers.

  1. 2023Finite-Time Analysis of Whittle Index based Q-Learning for Restless Multi-Armed Bandits with Neural Network Function Approximationcited by 14
  2. 2024Tabular and Deep Learning for the Whittle Indexcited by 3
  3. 2024Faster Q-Learning Algorithms for Restless Banditscited by 2
  4. 2024Lagrangian Index Policy for Restless Bandits with Average Rewardcited by 2
  5. 2025MARBLE: Multi-Armed Restless Bandits in Latent Markovian Environmentcited by 0