# Flatland-RL : Multi-Agent Reinforcement Learning on Trains

arXiv: https://arxiv.org/abs/2012.05893 · HTML: https://mlsanity.com/paper/2012.05893/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Dec 2020.

## Built on by (most cited first, 30 of 32)

- [Learning to run a Power Network Challenge: a Retrospective Analysis](https://mlsanity.com/paper/2103.03104/) (2021)
- [Towards a Standardised Performance Evaluation Protocol for Cooperative MARL](https://mlsanity.com/paper/2209.10485/) (2022)
- [MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learned](https://mlsanity.com/paper/2202.10583/) (2022)
- [Scalable Multi-Agent Model-Based Reinforcement Learning](https://mlsanity.com/paper/2205.15023/) (2022)
- [POGEMA: A Benchmark Platform for Cooperative Multi-Agent Pathfinding](https://mlsanity.com/paper/2407.14931/) (2024)
- [An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobility](https://mlsanity.com/paper/2203.07676/) (2022)
- [FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2406.02081/) (2024)
- [MAVIPER: Learning Decision Tree Policies for Interpretable Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2205.12449/) (2022)
- [MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2506.14990/) (2025)
- [Flatland Competition 2020: MAPF and MARL for Efficient Train Coordination on a Grid World](https://mlsanity.com/paper/2103.16511/) (2021)
- [IMP-MARL: a Suite of Environments for Large-scale Infrastructure Management Planning via MARL](https://mlsanity.com/paper/2306.11551/) (2023)
- [Off-the-Grid MARL: Datasets with Baselines for Offline Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2302.00521/) (2023)
- [Advancing MAPF Toward the Real World: A Scalable Multi-Agent Realistic Testbed (SMART)](https://mlsanity.com/paper/2503.04798/) (2025)
- [MABL: Bi-Level Latent-Variable World Model for Sample-Efficient Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2304.06011/) (2023)
- [Putting Data at the Centre of Offline Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2409.12001/) (2024)
- [Where Paths Collide: A Comprehensive Survey of Classic and Learning-Based Multi-Agent Pathfinding](https://mlsanity.com/paper/2505.19219/) (2025)
- [EvoCurr: Self-evolving Curriculum with Behavior Code Generation for Complex Decision-making](https://mlsanity.com/paper/2508.09586/) (2025)
- [Towards robust and domain agnostic reinforcement learning competitions](https://mlsanity.com/paper/2106.03748/) (2021)
- [Mitigating the Stability-Plasticity Dilemma in Adaptive Train Scheduling with Curriculum-Driven Continual DQN Expansion](https://mlsanity.com/paper/2408.09838/) (2024)
- [Multi-Agent Path Finding via Tree LSTM](https://mlsanity.com/paper/2210.12933/) (2022)
- [Airlift Challenge: A Competition for Optimizing Cargo Delivery](https://mlsanity.com/paper/2404.17716/) (2024)
- [Dynamic Pricing in High-Speed Railways Using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2501.08234/) (2025)
- [Toward Adaptable Multi-Agent Reinforcement Learning: An Assumption-Aware Review](https://mlsanity.com/paper/2507.10142/) (2025)
- [Standardized feature extraction from pairwise conflicts applied to the train rescheduling problem](https://mlsanity.com/paper/2204.03061/) (2022)
- [Terraforming -- Environment Manipulation during Disruptions for Multi-Agent Pickup and Delivery](https://mlsanity.com/paper/2305.11510/) (2023)
- [Multi-Agent Environments for Vehicle Routing Problems](https://mlsanity.com/paper/2411.14411/) (2024)
- [Opportunities of Reinforcement Learning in South Africa's Just Transition](https://mlsanity.com/paper/2411.15145/) (2024)
- [StarCraft+: Benchmarking Multi-agent Algorithms in Adversary Paradigm](https://mlsanity.com/paper/2512.16444/) (2025)
- [GRACE: A Unified 2D Multi-Robot Path Planning Simulator & Benchmark for Grid, Roadmap, And Continuous Environments](https://mlsanity.com/paper/2603.10858/) (2026)
- [Robust Asynchronous Planning via Auto-Formalization](https://mlsanity.com/paper/2606.00981/) (2026)

## Abstract (arXiv)

Efficient automated scheduling of trains remains a major challenge for modern railway systems. The underlying vehicle rescheduling problem (VRSP) has been a major focus of Operations Research (OR) since decades. Traditional approaches use complex simulators to study VRSP, where experimenting with a broad range of novel ideas is time consuming and has a huge computational overhead. In this paper, we introduce a two-dimensional simplified grid environment called "Flatland" that allows for faster experimentation. Flatland does not only reduce the complexity of the full physical simulation, but also provides an easy-to-use interface to test novel approaches for the VRSP, such as Reinforcement Learning (RL) and Imitation Learning (IL). In order to probe the potential of Machine Learning (ML) research on Flatland, we (1) ran a first series of RL and IL experiments and (2) design and executed a public Benchmark at NeurIPS 2020 to engage a large community of researchers to work on this problem. Our own experimental results, on the one hand, demonstrate that ML has potential in solving the VRSP on Flatland. On the other hand, we identify key topics that need further research. Overall, the Flatland environment has proven to be a robust and valuable framework to investigate the VRSP for railway networks. Our experiments provide a good starting point for further research and for the participants of the NeurIPS 2020 Flatland Benchmark. All of these efforts together have the potential to have a substantial impact on shaping the mobility of the future.