mlsanityData to 14 Oct 2026

Q-learning as a monotone scheme

arXiv 2405.20538 ↗First posted 30 May 2024cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Stability issues with reinforcement learning methods persist. To better understand some of these stability and convergence issues involving deep reinforcement learning methods, we examine a simple linear quadratic example. We interpret the convergence criterion of exact Q-learning in the sense of a monotone scheme and discuss consequences of function approximation on monotonicity properties.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 1 of its 12 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

1 of 12 references.

References · 1

  1. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 151

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Is Q-learning an Ill-posed Problem?
  2. 2026Heavy-Ball Q-Learning with Residual Weighting Correction
  3. 2026Periodic Regularized Q-Learning
  4. 2025Beyond Monotonicity: Revisiting Factorization Principles in Multi-Agent Q-Learning
  5. 2025Convergence and stability of Q-learning in Hierarchical Reinforcement Learning
  6. 2025Linear Q-Learning Does Not Diverge in L2: Convergence Rates to a Bounded Set
  7. 2026Target Updates May Stabilize Linear Q-Learning: Periodic and Soft Dynamics
  8. 2020Q* Approximation Schemes for Batch Reinforcement Learning: A Theoretical Comparison