mlsanityData to 7 Oct 2026

Full Gradient DQN Reinforcement Learning: A Provably Convergent Scheme

arXiv 2103.05981 ↗First posted 10 Mar 2021cs.LG math.OC math.PRcited by 8

Not on the map. Why?

Abstract (arXiv)

We analyze the DQN reinforcement learning algorithm as a stochastic approximation scheme using the o.d.e. (for 'ordinary differential equation') approach and point out certain theoretical issues. We then propose a modified scheme called Full Gradient DQN (FG-DQN, for short) that has a sound theoretical basis and compare it with the original scheme on sample problems. We observe a better performance for FG-DQN.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 14 of its 62 references that are in the map. First posted on arXiv 10 Mar 2021.

Builds on

14 of 62 references.

References · 14

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  3. 2016OpenAI Gymcited by 1,684
  4. 2015Prioritized Experience Replaycited by 1,070
  5. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 422
  6. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  7. 2018An Introduction to Deep Reinforcement Learningcited by 201
  8. 2019On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shiftcited by 174
  9. 2018Applications of Deep Reinforcement Learning in Communications and Networking: A Surveycited by 163
  10. 2017Deep Reinforcement Learning for De-Novo Drug Designcited by 161
  11. 2018Learning to Walk via Deep Reinforcement Learningcited by 153
  12. 2020Global optimality of softmax policy gradient with single hidden layer neural networks in the mean-field regimecited by 17
2 more
  1. 2016Analysis of gradient descent methods with non-diminishing, bounded errorscited by 8
  2. 2016Stochastic Recursive Inclusions with Non-Additive Iterate-Dependent Markov Noisecited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 2; 2025: 0; 2026: 2.

8 papers.

  1. 2023Stability of Q-Learning Through Design and Optimismcited by 7
  2. 2023Full Gradient Deep Reinforcement Learning for Average-Reward Criterioncited by 5
  3. 2023Convex Q Learning in a Stochastic Environment: Extended Versioncited by 5
  4. 2022Does DQN Learn?cited by 3
  5. 2024Faster Q-Learning Algorithms for Restless Banditscited by 2
  6. 2024Model Predictive Control is Almost Optimal for Restless Banditcited by 1
  7. 2026Optimistic Training and Convergence of Q-Learning -- Extended Versioncited by 1
  8. 2026Full-Gradient Successor Feature Representationscited by 0