mlsanityData to 14 Oct 2026

Where to Intervene: Action Selection in Deep Reinforcement Learning

arXiv 2507.04187 ↗First posted 5 Jul 2025stat.ML cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Deep reinforcement learning (RL) has gained widespread adoption in recent years but faces significant challenges, particularly in unknown and complex environments. Among these, high-dimensional action selection stands out as a critical problem. Existing works often require a sophisticated prior design to eliminate redundancy in the action space, relying heavily on domain expert experience or involving high computational complexity, which limits their generalizability across different RL tasks. In this paper, we address these challenges by proposing a general data-driven action selection approach with model-free and computationally friendly properties. Our method not only selects minimal sufficient actions but also controls the false discovery rate via knockoff sampling. More importantly, we seamlessly integrate the action selection into deep RL methods during online training. Empirical experiments validate the established theoretical guarantees, demonstrating that our method surpasses various alternative techniques in terms of both performance in variable selection and overall achieved rewards.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 36 of its 63 references that are in the map. First posted on arXiv 5 Jul 2025.

Builds on

36 of 63 references.

References · 36

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2019Model-Based Reinforcement Learning for Ataricited by 502
  5. 2019Reinforcement Learning in Healthcare: A Surveycited by 310
  6. 2016Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selectioncited by 121
  7. 2019Kinematic State Abstraction and Provably Efficient Rich-Observation Reinforcement Learningcited by 114
  8. 2014Controlling the false discovery rate via knockoffscited by 97
  9. 2019Concrete Autoencoders for Differentiable Feature Selection and Reconstructioncited by 90
  10. 2018Learn What Not to Learn: Action Elimination with Deep Reinforcement Learningcited by 77
  11. 2020Smooth Exploration for Robotic Reinforcement Learningcited by 54
  12. 2020Action Space Shaping in Deep Reinforcement Learningcited by 46
24 more
  1. 2018Deep Knockoffscited by 41
  2. 2019Growing Action Spacescited by 29
  3. 2022DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systemscited by 29
  4. 2021Estimating Optimal Infinite Horizon Dynamic Treatment Regimes via pT-Learningcited by 26
  5. 2020Sparse Feature Selection Makes Batch Reinforcement Learning More Sample Efficientcited by 24
  6. 2018Deep Reinforcement Learning for Dynamic Treatment Regimes on Medical Registry Datacited by 23
  7. 2019Synthesis of Biologically Realistic Human Motion Using Joint Torque Actuationcited by 23
  8. 2023Latent Exploration for Reinforcement Learningcited by 22
  9. 2017Kernel Feature Selection via Conditional Covariance Minimizationcited by 20
  10. 2023Towards Trustworthy Explanation: On Causal Rationalizationcited by 19
  11. 2020FRESH: Interactive Reward Shaping in High-Dimensional State Spaces using Human Feedbackcited by 15
  12. 2019Algorithmic Improvements for Deep Reinforcement Learning applied to Interactive Fictioncited by 14
  13. 2020Deep Jump Learning for Off-Policy Evaluation in Continuous Treatment Settingscited by 14
  14. 2023On Learning Necessary and Sufficient Causal Graphscited by 14
  15. 2015Deep Reinforcement Learning with Attention for Slate Markov Decision Processes with High-Dimensional States and Actionscited by 13
  16. 2018Auto-Encoding Knockoff Generator for FDR Controlled Variable Selectioncited by 8
  17. 2022Scaling Marginalized Importance Sampling to High-Dimensional State-Spaces via State Abstractioncited by 7
  18. 2023Quasi-optimal Reinforcement Learning with Continuous Actionscited by 7
  19. 2023Sequential Knockoffs for Variable Selection in Reinforcement Learningcited by 7
  20. 2018A Parsimonious Personalized Dose Finding Model via Dimension Reductioncited by 6
  21. 2023No Prior Mask: Eliminate Redundant Action for Deep Reinforcement Learningcited by 6
  22. 2017Sample Efficient Feature Selection for Factored MDPscited by 5
  23. 2019Fast Online Reinforcement Learning Control using State-Space Dimensionality Reductioncited by 1
  24. 2021A Generalized Knockoff Procedure for FDR Control in Structural Change Detectioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025RL-Selector: Reinforcement Learning-Guided Data Selection via Redundancy Assessment
  2. 2025Improved Training Mechanism for Reinforcement Learning via Online Model Selection
  3. 2024Efficient Exploration in Deep Reinforcement Learning: A Novel Bayesian Actor-Critic Algorithm
  4. 2024Action Mapping for Reinforcement Learning in Continuous Environments with Constraints
  5. 2026GEM: Guided Expectation-Maximization for Behavior-Normalized Candidate Action Selection in Offline RL
  6. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrations
  7. 2024Learning Rate-Free Reinforcement Learning: A Case for Model Selection with Non-Stationary Objectives
  8. 2025Towards General-Purpose Model-Free Reinforcement Learning