Geometrically Averaged Hard Target Updates for Linear Q-Learning
On the map: Deep reinforcement learning methods → RL theory and sample complexity
Abstract (arXiv)
Periodic hard target updates are among the most common stabilization devices in modern deep Q-learning. Recent studies suggest that target updates can improve stability in Q-learning with function approximation, including linear function approximation. We introduce and analyze the so-called λ-target update, obtained by averaging the m-periodic target update maps with λ-geometric weights (1-λ)λm-1, λ ∈ [0,1]. The endpoint λ=0 recovers the one-period target update, while the continuous endpoint λuparrow1 recovers projected Q-value iteration. We study this mechanism for Q-learning with linear function approximation, namely linear Q-learning, using a switching-system model and related tools. For clarity, the paper treats a deterministic version; the formulation extends to stochastic reinforcement-learning settings.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 38 references that are in the map. First posted on arXiv 9 Jun 2026.
Builds on
12 of 38 references.
Closest ancestors · 4
Same direction, newest first
- 2026Target Updates May Stabilize Linear Q-Learning: Periodic and Soft Dynamics31 May
- 2026A Switching System Theory of Q-Learning with Linear Function Approximation10 May
- 2026Switching Theory for Q-Learning21 Apr
- 2025Understanding the theoretical properties of projected Bellman equation, linear Q-learning, and approximate value iteration15 Apr
Other references · 8
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2022Target Network and Truncation Overcome The Deadly Triad in Q-Learningcited by 31
- 2019Target-Based Temporal Difference Learningcited by 20
- 2021A Discrete-Time Switching System Analysis of Q-learningcited by 16
- 2022Regularized Q-learningcited by 16
- 2023Why Target Networks Stabilise Temporal Difference Methodscited by 13
- 2020Periodic Q-Learningcited by 10
- 2024Target Networks and Over-parameterization Stabilize Off-policy Bootstrapping with Function Approximationcited by 10
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026The Role of Target Update Frequencies in Q-Learning
- 2026Heavy-Ball Q-Learning with Residual Weighting Correction
- 2025From Set Convergence to Pointwise Convergence: Finite-Time Guarantees for Average-Reward Q-Learning with Adaptive Stepsizes
- 2026Q-Learning with Stable Infinite-Dimensional Linear Function Approximation
- 2025Deep Q-Learning with Gradient Target Tracking
- 2026Periodic Regularized Q-Learning
- 2025Linear Q-Learning Does Not Diverge in L2: Convergence Rates to a Bounded Set
- 2026Deep Q-Learning on H"older Spaces