mlsanityData to 14 Oct 2026

A Closer Look at Invalid Action Masking in Policy Gradient Algorithms

arXiv 2006.14171 ↗First posted 25 Jun 2020cs.LG cs.AI stat.MLcited by 171

Not on the map. Why?

Cited by 80 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In recent years, Deep Reinforcement Learning (DRL) algorithms have achieved state-of-the-art performance in many challenging strategy games. Because these games have complicated rules, an action sampled from the full discrete action distribution predicted by the learned policy is likely to be invalid according to the game rules (e.g., walking into a wall). The usual approach to deal with this problem in policy gradient algorithms is to "mask out" invalid actions and just sample from the set of valid actions. The implications of this process, however, remain under-investigated. In this paper, we 1) show theoretical justification for such a practice, 2) empirically demonstrate its importance as the space of invalid actions grows, and 3) provide further insights by evaluating different action masking regimes, such as removing masking after an agent has been trained using masking. The source code can be found at https://github.com/vwxyzjn/invalid-action-masking

Key facts. Cited by 171 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 11; 2023: 19; 2024: 31; 2025: 49; 2026: 53.

171 papers.

Sort:

By area:

  1. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 246
  2. 2023Generating Synergistic Formulaic Alpha Collections via Reinforcement Learningcited by 39
  3. 2021Understanding Continual Learning Settings with Data Distribution Drift Analysiscited by 38
  4. 2022Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarkingcited by 34
  5. 2022Scalable Multi-Agent Reinforcement Learning for Warehouse Logistics with Robotic and Human Co-Workerscited by 30
  6. 2024AlphaForge: A Framework to Mine and Dynamically Combine Formulaic Alpha Factorscited by 27
  7. 2022Provably Safe Reinforcement Learning via Action Projection using Reachability Analysis and Polynomial Zonotopescited by 26
  8. 2022Beyond CAGE: Investigating Generalization of Learned Autonomous Network Defense Policiescited by 23
  9. 2021A Reinforcement Learning Environment For Job-Shop Schedulingcited by 21
  10. 2021Gym-μRTS: Toward Affordable Full Game Real-time Strategy Games Research with Deep Reinforcement Learningcited by 19
  11. 2024Excluding the Irrelevant: Focusing Reinforcement Learning through Continuous Action Maskingcited by 19
  12. 2023Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulationcited by 14
38 more
  1. 2024HOPE: A Reinforcement Learning-based Hybrid Policy Path Planner for Diverse Parking Scenarioscited by 11
  2. 2023Generative Modelling of Stochastic Actions with Arbitrary Constraints in Reinforcement Learningcited by 10
  3. 2021Generalising Discrete Action Spaces with Conditional Action Treescited by 6
  4. 2022Safe and Psychologically Pleasant Traffic Signal Control with Reinforcement Learning using Action Maskingcited by 6
  5. 2024Introducing PetriRL: An Innovative Framework for JSSP Resolution Integrating Petri nets and Event-based Reinforcement Learningcited by 6
  6. 2021Deep Reinforcement Learning for Constrained Field Development Optimization in Subsurface Two-phase Flowcited by 5
  7. 2022Learning to design without prior data: Discovering generalizable design strategies using deep learning and tree searchcited by 5
  8. 2023Discovery of Optimal Quantum Error Correcting Codes via Reinforcement Learningcited by 5
  9. 2024Physics-Aware Combinatorial Assembly Sequence Planning using Data-free Action Maskingcited by 5
  10. 2024Effective Analog ICs Floorplanning with Relational Graph Neural Networks and Reinforcement Learningcited by 5
  11. 2024Action Mapping for Reinforcement Learning in Continuous Environments with Constraintscited by 5
  12. 2025Discovering highly efficient low-weight quantum error-correcting codes with reinforcement learningcited by 5
  13. 2023Learning policies for resource allocation in business processescited by 4
  14. 2025Reinforced Context Order Recovery for Adaptive Reasoning and Planningcited by 4
  15. 2021CleanRL: High-quality Single-file Implementations of Deep Reinforcement Learning Algorithmscited by 3
  16. 2023Learning to Recharge: UAV Coverage Path Planning through Deep Reinforcement Learningcited by 3
  17. 2024Scaling Team Coordination on Graphs with Reinforcement Learningcited by 3
  18. 2024Revisiting Space Mission Planning: A Reinforcement Learning-Guided Approach for Multi-Debris Rendezvouscited by 3
  19. 2024BraVE: Offline Reinforcement Learning for Discrete Combinatorial Action Spacescited by 3
  20. 2025Increasing Interaction Fidelity: Training Routines for Biomechanical Models in HCIcited by 3
  21. 2025Narrowing Action Choices with AI Improves Human Sequential Decisionscited by 3
  22. 2021Reinforcement Learning With Sparse-Executing Actions via Sparsity Regularizationcited by 2
  23. 2022A Reinforcement Learning Approach to Domain-Knowledge Inclusion Using Grammar Guided Symbolic Regressioncited by 2
  24. 2023Learning to Generate All Feasible Actionscited by 2
  25. 2023Multi-Agent Reinforcement Learning with Action Masking for UAV-enabled Mobile Communicationscited by 2
  26. 2023X-RLflow: Graph Reinforcement Learning for Neural Network Subgraphs Transformationcited by 2
  27. 2024A Competition Winning Deep Reinforcement Learning Agent in microRTScited by 2
  28. 2024Climate Adaptation with Reinforcement Learning: Experiments with Flooding and Transportation in Copenhagencited by 2
  29. 2024From Imitation to Exploration: End-to-end Autonomous Driving based on World Modelcited by 2
  30. 2024Multi-Agent Actor-Critics in Autonomous Cyber Defensecited by 2
  31. 2024Integrating Transit Signal Priority into Multi-Agent Reinforcement Learning based Traffic Signal Controlcited by 2
  32. 2025Embodied Escaping: End-to-End Reinforcement Learning for Robot Navigation in Narrow Environmentcited by 2
  33. 2025Optimizing Navigation And Chemical Application in Precision Agriculture With Deep Reinforcement Learning And Conditional Action Treecited by 2
  34. 2025Graph-Enhanced Model-Free Reinforcement Learning Agents for Efficient Power Grid Topological Controlcited by 2
  35. 2025Integrating Human Knowledge Through Action Masking in Reinforcement Learning for Operations Researchcited by 2
  36. 2025pix2pockets: Shot Suggestions in 8-Ball Pool from a Single Image in the Wildcited by 2
  37. 2025Delphos: A reinforcement learning framework for assisting discrete choice model specificationcited by 2
  38. 2025Goal-Conditioned Reinforcement Learning for Data-Driven Maritime Navigationcited by 2

50 most cited of 171 All 171 (JSON)