# Geometrically Averaged Hard Target Updates for Linear Q-Learning

arXiv: https://arxiv.org/abs/2606.10835 · HTML: https://mlsanity.com/paper/2606.10835/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 38 references that are in the map.
- First posted on arXiv 9 Jun 2026.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 12 of 12)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Target Network and Truncation Overcome The Deadly Triad in Q-Learning](https://mlsanity.com/paper/2203.02628/) (2022)
- [Target-Based Temporal Difference Learning](https://mlsanity.com/paper/1904.10945/) (2019)
- [A Discrete-Time Switching System Analysis of Q-learning](https://mlsanity.com/paper/2102.08583/) (2021)
- [Regularized Q-learning](https://mlsanity.com/paper/2202.05404/) (2022)
- [Why Target Networks Stabilise Temporal Difference Methods](https://mlsanity.com/paper/2302.12537/) (2023)
- [Periodic Q-Learning](https://mlsanity.com/paper/2002.09795/) (2020)
- [Target Networks and Over-parameterization Stabilize Off-policy Bootstrapping with Function Approximation](https://mlsanity.com/paper/2405.21043/) (2024)
- [Switching Theory for Q-Learning](https://mlsanity.com/paper/2604.19569/) (2026)
- [Understanding the theoretical properties of projected Bellman equation, linear Q-learning, and approximate value iteration](https://mlsanity.com/paper/2504.10865/) (2025)
- [A Switching System Theory of Q-Learning with Linear Function Approximation](https://mlsanity.com/paper/2605.11021/) (2026)
- [Target Updates May Stabilize Linear Q-Learning: Periodic and Soft Dynamics](https://mlsanity.com/paper/2606.02645/) (2026)

## Abstract (arXiv)

Periodic hard target updates are among the most common stabilization devices in modern deep Q-learning. Recent studies suggest that target updates can improve stability in Q-learning with function approximation, including linear function approximation. We introduce and analyze the so-called λ-target update, obtained by averaging the m-periodic target update maps with λ-geometric weights (1-λ)λm-1, λ ∈ [0,1]. The endpoint λ=0 recovers the one-period target update, while the continuous endpoint λuparrow1 recovers projected Q-value iteration. We study this mechanism for Q-learning with linear function approximation, namely linear Q-learning, using a switching-system model and related tools. For clarity, the paper treats a deterministic version; the formulation extends to stochastic reinforcement-learning settings.