# A Reinforcement Learning Environment For Job-Shop Scheduling

arXiv: https://arxiv.org/abs/2104.03760 · HTML: https://mlsanity.com/paper/2104.03760/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 25 references that are in the map.
- First posted on arXiv 8 Apr 2021.

## Builds on (most cited first, 7 of 7)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon](https://mlsanity.com/paper/1811.06128/) (2018)
- [A Closer Look at Invalid Action Masking in Policy Gradient Algorithms](https://mlsanity.com/paper/2006.14171/) (2020)
- [Learning to Dispatch for Job Shop Scheduling via Deep Reinforcement Learning](https://mlsanity.com/paper/2010.12367/) (2020)

## Built on by (most cited first, 21 of 21)

- [Deep Reinforcement Learning Guided Improvement Heuristic for Job Shop Scheduling](https://mlsanity.com/paper/2211.10936/) (2022)
- [Graph Neural Networks for Job Shop Scheduling Problems: A Survey](https://mlsanity.com/paper/2406.14096/) (2024)
- [An End-to-End Reinforcement Learning Approach for Job-Shop Scheduling Problems Based on Constraint Programming](https://mlsanity.com/paper/2306.05747/) (2023)
- [Learning Topological Representations with Bidirectional Graph Attention Network for Solving Job Shop Scheduling Problem](https://mlsanity.com/paper/2402.17606/) (2024)
- [Decomposition Strategies and Multi-shot ASP Solving for Job-shop Scheduling](https://mlsanity.com/paper/2205.07537/) (2022)
- [Policy-Based Self-Competition for Planning Problems](https://mlsanity.com/paper/2306.04403/) (2023)
- [Solving the flexible job-shop scheduling problem through an enhanced deep reinforcement learning approach](https://mlsanity.com/paper/2310.15706/) (2023)
- [Attention-based Reinforcement Learning for Combinatorial Optimization: Application to Job Shop Scheduling Problem](https://mlsanity.com/paper/2401.16580/) (2024)
- [Scalable Production Scheduling: Linear Complexity via Unified Homogeneous Graphs](https://mlsanity.com/paper/2604.23841/) (2026)
- [A Production Scheduling Framework for Reinforcement Learning Under Real-World Constraints](https://mlsanity.com/paper/2506.13566/) (2025)
- [Semiconductor Fab Scheduling with Self-Supervised and Reinforcement Learning](https://mlsanity.com/paper/2302.07162/) (2023)
- [A Reinforcement Learning Approach for Scheduling Problems With Improved Generalization Through Order Swapping](https://mlsanity.com/paper/2302.13941/) (2023)
- [Graph Neural Networks for the Offline Nanosatellite Task Scheduling Problem](https://mlsanity.com/paper/2303.13773/) (2023)
- [Decision Transformer for Enhancing Neural Local Search on the Job Shop Scheduling Problem](https://mlsanity.com/paper/2409.02697/) (2024)
- [Optimizing Job Shop Scheduling in the Furniture Industry: A Reinforcement Learning Approach Considering Machine Setup, Batch Variability, and Intralogistics](https://mlsanity.com/paper/2409.11820/) (2024)
- [Scalability of Reinforcement Learning Methods for Dispatching in Semiconductor Frontend Fabs: A Comparison of Open-Source Models with Real Industry Datasets](https://mlsanity.com/paper/2505.11135/) (2025)
- [Unraveling the Rainbow: can value-based methods schedule?](https://mlsanity.com/paper/2505.03323/) (2025)
- [Deep Learning for Unrelated-Machines Scheduling: Handling Variable Dimensions](https://mlsanity.com/paper/2512.19527/) (2025)
- [Mamba Meets Scheduling: Learning to Solve Flexible Job Shop Scheduling with Efficient Sequence Modeling](https://mlsanity.com/paper/2602.21546/) (2026)
- [HGT-Scheduler: Deep Reinforcement Learning for the Job Shop Scheduling Problem via Heterogeneous Graph Transformers](https://mlsanity.com/paper/2603.06777/) (2026)
- [Curriculum Learning with GNN-based Reinforcement Learning for Job Shop Scheduling](https://mlsanity.com/paper/2609.28085/) (2026)

## Abstract (arXiv)

Scheduling is a fundamental task occurring in various automated systems applications, e.g., optimal schedules for machines on a job shop allow for a reduction of production costs and waste. Nevertheless, finding such schedules is often intractable and cannot be achieved by Combinatorial Optimization Problem (COP) methods within a given time limit. Recent advances of Deep Reinforcement Learning (DRL) in learning complex behavior enable new COP application possibilities. This paper presents an efficient DRL environment for Job-Shop Scheduling -- an important problem in the field. Furthermore, we design a meaningful and compact state representation as well as a novel, simple dense reward function, closely related to the sparse make-span minimization criteria used by COP methods. We demonstrate that our approach significantly outperforms existing DRL methods on classic benchmark instances, coming close to state-of-the-art COP approaches.