mlsanityData to 14 Oct 2026

Selective Growing Q-Networks (SGQN): An Analysis of Adaptive Action Grids in Decoupled Q-Learning

arXiv 2610.20463 ↗First posted 14 Oct 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations also fits: RL theory and sample complexity

Abstract (arXiv)

Value-based Q-learning for continuous control can discretise the action space, but the resolution of the discretisation trades early exploration against final precision. Growing Q-Networks (GQN) grow the resolution uniformly from coarse to fine. We introduce Selective Growing Q-Networks (SGQN), which keeps decoupled Q-learning and replaces uniform growth with an adaptive action grid: bins are added next to bins with high estimated Q-value, bins with low Q-value are pruned, and actions are chosen epsilon-greedily. On eight DeepMind Control Suite (DMControl) tasks with 2 to 12 action dimensions, SGQN has the highest mean final return among the compared methods on four, is within 1.1% of the best method on three more, and is 3.4% below Bang-Bang MPO on Walker Run, using on average 7.5 of 9 available bins per dimension. We then test whether the adaptive grid explains these results, and find that it does not. Growth activates every candidate bin and behaves like uniform growth, and removing pruning changes the return by at most 1.1%. Two other choices matter: starting from a coarse grid, since starting with all 9 bins produces collapsed seeds, and epsilon-greedy selection, since an annealed softmax lowers the return by 23% to 100%. SGQN is insensitive to its growth and pruning hyperparameters. It also differs from the GQN and DecQN baselines in other respects, notably one gradient update per environment step instead of one per eight, which we did not control for, so we do not attribute its advantage to a single component. We present SGQN as a competitive value-based method and as an analysis of which ingredients of a growing action grid matter.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 18 of its 22 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

18 of 22 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Continuous Control with Coarse-to-fine Reinforcement Learning10 Jul

Other references · 17

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2016Layer Normalizationcited by 5,669
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  6. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  7. 2015Prioritized Experience Replaycited by 1,099
  8. 2018DeepMind Control Suitecited by 956
  9. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  10. 2018Maximum a Posteriori Policy Optimisationcited by 328
  11. 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 157
  12. 2017Action Branching Architectures for Deep Reinforcement Learningcited by 92
5 more
  1. 2019Discretizing Continuous Action Space for On-Policy Optimizationcited by 68
  2. 2021Is Bang-Bang Control All You Need? Solving Continuous Control with Bernoulli Policiescited by 38
  3. 2018Mix&Match - Agent Curricula for Reinforcement Learningcited by 27
  4. 2022Solving Continuous Control via Q-learningcited by 24
  5. 2024Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolutioncited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025GB-DQN: Gradient Boosted DQN Models for Non-stationary Reinforcement Learning
  2. 2024Coarse-to-fine Q-Network with Action Sequence for Data-Efficient Reinforcement Learning
  3. 2025Enhancing Q-Value Updates in Deep Q-Learning via Successor-State Prediction
  4. 2024Adaptive Q-Network: On-the-fly Target Selection for Deep Reinforcement Learning
  5. 2025Learning from Suboptimal Data in Continuous Control via Auto-Regressive Soft Q-Network
  6. 2025Growing with Experience: Growing Neural Networks in Deep Reinforcement Learning
  7. 2024Learning in complex action spaces without policy gradients
  8. 2025β-DQN: Improving Deep Q-Learning By Evolving the Behavior