mlsanityData to 14 Oct 2026

Breaking the Grid: Distance-Guided Reinforcement Learning in Large Discrete Action Spaces

arXiv 2602.08616 ↗First posted 9 Feb 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Reinforcement Learning (RL) is increasingly applied to large-scale decision-making problems like logistics, scheduling, and recommender systems, but existing algorithms struggle with the curse of dimensionality in such large discrete action spaces. We propose Distance-Guided Reinforcement Learning (DGRL), combining Sampled Dynamic Neighborhoods and Distance-Based Updates to enable efficient RL in problems with up to 1020 actions. Unlike prior methods, DGRL performs stochastic volumetric exploration and transforms policy optimization into a stable regression task, decoupling gradient variance from action space cardinality. On structured tasks, DGRL provably guarantees local value improvement. DGRL naturally generalizes to hybrid continuous-discrete action spaces. We demonstrate performance improvements of up to 66% against state-of-the-art benchmarks across regularly and irregularly structured environments, while simultaneously improving convergence speed and computational complexity.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 15 of its 31 references that are in the map. First posted on arXiv 9 Feb 2026.

Builds on

15 of 31 references.

References · 15

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 574
  3. 2018Maximum a Posteriori Policy Optimisationcited by 328
  4. 2015Deep Reinforcement Learning in Large Discrete Action Spacescited by 161
  5. 2019Learning Action Representations for Reinforcement Learningcited by 75
  6. 2018Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Spacecited by 58
  7. 2019Hybrid Actor-Critic Reinforcement Learning in Parameterized Action Spacecited by 54
  8. 2019Discrete and Continuous Action Representation for Practical RL in Video Gamescited by 39
  9. 2021HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representationcited by 35
  10. 2019Dynamics-aware Embeddingscited by 34
  11. 2019Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spacescited by 24
  12. 2018Hierarchical Approaches for Reinforcement Learning in Parameterized Action Spacecited by 15
3 more
  1. 2025Structured Reinforcement Learning for Combinatorial Decision-Makingcited by 7
  2. 2023Dynamic Neighborhood Construction for Structured Large Discrete Action Spacescited by 4
  3. 2024Mitigating Suboptimality of Deterministic Policy Gradients in Complex Q-functionscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Efficient Exploration in Deep Reinforcement Learning: A Novel Bayesian Actor-Critic Algorithm
  2. 2024Trajectory-Oriented Policy Optimization with Sparse Rewards
  3. 2025LLM-Guided Reinforcement Learning: Addressing Training Bottlenecks through Policy Modulation
  4. 2026Trajectory-Aware Policy Optimization: Closing the State Distribution Gap in Long-Horizon LLM RL
  5. 2026Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learning
  6. 2026Mesh-RL: Coupled subgrid reinforcement learning
  7. 2025Context-Sensitive Abstractions for Reinforcement Learning with Parameterized Actions
  8. 2026Improving and Accelerating Offline RL in Large Discrete Action Spaces with Structured Policy Initialization