Integrating Human Knowledge Through Action Masking in Reinforcement Learning for Operations Research
On the map: Deep reinforcement learning methods → RL training of language and agent models
Abstract (arXiv)
Reinforcement learning (RL) provides a powerful method to address problems in operations research. However, its real-world application often fails due to a lack of user acceptance and trust. A possible remedy is to provide managers with the possibility of altering the RL policy by incorporating human expert knowledge. In this study, we analyze the benefits and caveats of including human knowledge via action masking. While action masking has so far been used to exclude invalid actions, its ability to integrate human expertise remains underexplored. Human knowledge is often encapsulated in heuristics, which suggest reasonable, near-optimal actions in certain situations. Enforcing such actions should hence increase trust among the human workforce to rely on the model's decisions. Yet, a strict enforcement of heuristic actions may also restrict the policy from exploring superior actions, thereby leading to overall lower performance. We analyze the effects of action masking based on three problems with different characteristics, namely, paint shop scheduling, peak load management, and inventory management. Our findings demonstrate that incorporating human knowledge through action masking can achieve substantial improvements over policies trained without action masking. In addition, we find that action masking is crucial for learning effective policies in constrained action spaces, where certain actions can only be performed a limited number of times. Finally, we highlight the potential for suboptimal outcomes when action masks are overly restrictive.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 10 of its 53 references that are in the map. First posted on arXiv 3 Apr 2025.
Builds on
10 of 53 references.
References · 10
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
- 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
- 2018Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizoncited by 623
- 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 298
- 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 274
- 2018Deep Reinforcement Learning and the Deadly Triadcited by 174
- 2020A Closer Look at Invalid Action Masking in Policy Gradient Algorithmscited by 171
- 2021Heuristic-Guided Reinforcement Learningcited by 42
- 2023Reinforcement and Deep Reinforcement Learning-based Solutions for Machine Maintenance Planning, Scheduling Policies, and Optimizationcited by 8
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2023Learning Complementary Policies for Human-AI Teams
- 2019Leveraging Human Guidance for Deep Reinforcement Learning Tasks
- 2026Improving Human Performance with Value-Aware Interventions: A Case Study in Chess
- 2024Heuristic Algorithm-based Action Masking Reinforcement Learning (HAAM-RL) with Ensemble Inference Method
- 2025Strategy Masking: A Method for Guardrails in Value-based Reinforcement Learning Agents
- 2024Unveiling the Role of Expert Guidance: A Comparative Analysis of User-centered Imitation Learning and Traditional Reinforcement Learning
- 2024Excluding the Irrelevant: Focusing Reinforcement Learning through Continuous Action Masking
- 2023Foundations for Transfer in Reinforcement Learning: A Taxonomy of Knowledge Modalities