# Scalable Multi-agent Reinforcement Learning for Factory-wide Dynamic Scheduling

arXiv: https://arxiv.org/abs/2409.13571 · HTML: https://mlsanity.com/paper/2409.13571/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 51 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Reinforcement learning](https://mlsanity.com/paper/2405.10369/) (2024)
- [Equivalence Between Policy Gradients and Soft Q-Learning](https://mlsanity.com/paper/1704.06440/) (2017)
- [A Benchmark Environment Motivated by Industrial Control Problems](https://mlsanity.com/paper/1709.09480/) (2017)
- [CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement Learning](https://mlsanity.com/paper/1809.05188/) (2018)
- [Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy Optimization](https://mlsanity.com/paper/1911.05478/) (2019)
- [Proximal Policy Optimization with Mixed Distributed Training](https://mlsanity.com/paper/1907.06479/) (2019)
- [Learning Multiple Coordinated Agents under Directed Acyclic Graph Constraints](https://mlsanity.com/paper/2307.07529/) (2023)

## Abstract (arXiv)

Real-time dynamic scheduling is a crucial but notoriously challenging task in modern manufacturing processes due to its high decision complexity. Recently, reinforcement learning (RL) has been gaining attention as an impactful technique to handle this challenge. However, classical RL methods typically rely on human-made dispatching rules, which are not suitable for large-scale factory-wide scheduling. To bridge this gap, this paper applies a leader-follower multi-agent RL (MARL) concept to obtain desired coordination after decomposing the scheduling problem into a set of sub-problems that are handled by each individual agent for scalability. We further strengthen the procedure by proposing a rule-based conversion algorithm to prevent catastrophic loss of production capacity due to an agent's error. Our experimental results demonstrate that the proposed model outperforms the state-of-the-art deep RL-based scheduling models in various aspects. Additionally, the proposed model provides the most robust scheduling performance to demand changes. Overall, the proposed MARL-based scheduling model presents a promising solution to the real-time scheduling problem, with potential applications in various manufacturing industries.