mlsanityData to 7 Oct 2026

Action Mapping for Reinforcement Learning in Continuous Environments with Constraints

arXiv 2412.04327 ↗First posted 5 Dec 2024cs.LG cs.AI cs.SYcited by 4

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (DRL) has had success across various domains, but applying it to environments with constraints remains challenging due to poor sample efficiency and slow convergence. Recent literature explored incorporating model knowledge to mitigate these problems, particularly through the use of models that assess the feasibility of proposed actions. However, integrating feasibility models efficiently into DRL pipelines in environments with continuous action spaces is non-trivial. We propose a novel DRL training strategy utilizing action mapping that leverages feasibility models to streamline the learning process. By decoupling the learning of feasible actions from policy optimization, action mapping allows DRL agents to focus on selecting the optimal action from a reduced feasible action set. We demonstrate through experiments that action mapping significantly improves training performance in constrained environments with continuous action spaces, especially with imperfect feasibility models.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 20 of its 36 references that are in the map. First posted on arXiv 5 Dec 2024.

Builds on

20 of 36 references.

References · 20

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  4. 2017Constrained Policy Optimizationcited by 928
  5. 2018Reinforcement Learning for Solving the Vehicle Routing Problemcited by 327
  6. 2018Safe Exploration in Continuous Action Spacescited by 237
  7. 2021DC3: A learning method for optimization with hard constraintscited by 181
  8. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  9. 2020A Closer Look at Invalid Action Masking in Policy Gradient Algorithmscited by 160
  10. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 140
  11. 2020Learning to be Safe: Deep RL with a Safety Criticcited by 109
  12. 2019Benchmarking Batch Deep Reinforcement Learning Algorithmscited by 108
8 more
  1. 2020Conservative Safety Critics for Explorationcited by 107
  2. 2020Learning to Walk in the Real World with Minimal Human Effortcited by 103
  3. 2022Penalized Proximal Policy Optimization for Safe Reinforcement Learningcited by 60
  4. 2023State-wise Safe Reinforcement Learning: A Surveycited by 48
  5. 2021HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representationcited by 33
  6. 2021Feasible Actor-Critic: Constrained Reinforcement Learning for Ensuring Statewise Safetycited by 30
  7. 2024Excluding the Irrelevant: Focusing Reinforcement Learning through Continuous Action Maskingcited by 17
  8. 2024RaceMOP: Mapless Online Path Planning for Multi-Agent Autonomous Racing using Residual Policy Learningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Beyond Distributions: Geometric Action Control for Continuous Reinforcement Learningcited by 1
  2. 2025Continuous World Coverage Path Planning for Fixed-Wing UAVs using Deep Reinforcement Learningcited by 0
  3. 2026Overcoming Valid Action Suppression in Unmasked Policy Gradient Algorithmscited by 0
  4. 2026Comfort by Construction: Adaptive, Comfort-Bounded Action Spaces for Learned Driving Policiescited by 0