# Advancing MAPF Toward the Real World: A Scalable Multi-Agent Realistic Testbed (SMART)

arXiv: https://arxiv.org/abs/2503.04798 · HTML: https://mlsanity.com/paper/2503.04798/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 36 references that are in the map.
- First posted on arXiv 3 Mar 2025.

## Builds on (most cited first, 16 of 16)

- [Multi-Agent Pathfinding: Definitions, Variants, and Benchmarks](https://mlsanity.com/paper/1906.08291/) (2019)
- [Searching with Consistent Prioritization for Multi-Agent Path Finding](https://mlsanity.com/paper/1812.06356/) (2018)
- [EECBS: A Bounded-Suboptimal Search for Multi-Agent Path Finding](https://mlsanity.com/paper/2010.01367/) (2020)
- [Lifelong Multi-Agent Path Finding for Online Pickup and Delivery Tasks](https://mlsanity.com/paper/1705.10868/) (2017)
- [Priority Inheritance with Backtracking for Iterative Multi-agent Path Finding](https://mlsanity.com/paper/1901.11282/) (2019)
- [LaCAM: Search-Based Algorithm for Quick Multi-Agent Pathfinding](https://mlsanity.com/paper/2211.13432/) (2022)
- [Flatland-RL : Multi-Agent Reinforcement Learning on Trains](https://mlsanity.com/paper/2012.05893/) (2020)
- [Feasibility Study: Moving Non-Homogeneous Teams in Congested Video Game Environments](https://mlsanity.com/paper/1710.01447/) (2017)
- [Experimenting with robotic intra-logistics domains](https://mlsanity.com/paper/1804.10247/) (2018)
- [Multi-Agent Motion Planning For Differential Drive Robots Through Stationary State Search](https://mlsanity.com/paper/2412.13359/) (2024)
- [Graph-Based Multi-Robot Path Finding and Planning](https://mlsanity.com/paper/2206.11319/) (2022)
- [Multi-Agent Motion Planning with B'ezier Curve Optimization under Kinodynamic Constraints](https://mlsanity.com/paper/2311.14145/) (2023)
- [Receding Horizon Re-ordering of Multi-Agent Execution Schedules](https://mlsanity.com/paper/2312.04190/) (2023)
- [A Real-Time Rescheduling Algorithm for Multi-robot Plan Execution](https://mlsanity.com/paper/2403.18145/) (2024)
- [Provably Safe and Deadlock-Free Execution of Multi-Robot Plans under Delaying Disturbances](https://mlsanity.com/paper/1603.08582/) (2016)
- [QD-MAPPER: A Quality Diversity Framework to Automatically Evaluate Multi-Agent Path Finding Algorithms in Diverse Maps](https://mlsanity.com/paper/2409.06888/) (2024)

## Built on by (most cited first, 5 of 5)

- [CAMAR: Continuous Actions Multi-Agent Routing](https://mlsanity.com/paper/2508.12845/) (2025)
- [WinkTPG: An Execution Framework for Multi-Agent Path Finding Using Temporal Reasoning](https://mlsanity.com/paper/2508.01495/) (2025)
- [Lifelong Scalable Multi-Agent Realistic Testbed and A Comprehensive Study on Design Choices in Lifelong AGV Fleet Management Systems](https://mlsanity.com/paper/2602.15721/) (2026)
- [From Discrete Plans to Real-World Execution: A World-Model-Driven Framework for Execution-Aware Multi-Agent Path Finding](https://mlsanity.com/paper/2511.21886/) (2025)
- [Analyzing Planner Design Trade-offs for MAPF under ADG-based Realistic Execution](https://mlsanity.com/paper/2512.09736/) (2025)

## Abstract (arXiv)

We present Scalable Multi-Agent Realistic Testbed (SMART), a realistic and efficient software tool for evaluating Multi-Agent Path Finding (MAPF) algorithms. MAPF focuses on planning collision-free paths for a group of robots. While state-of-the-art MAPF planners can plan paths for hundreds of robots in seconds, they often rely on simplified robot models, making their real-world performance unclear. Researchers typically lack access to hundreds of physical robots in laboratory settings to evaluate the algorithms. Meanwhile, industrial professionals who lack expertise in MAPF require an easy-to-use simulator to efficiently test and understand the performance of MAPF planners in their specific settings. SMART fills this gap with several advantages: (1) SMART uses physics-engine-based simulators to create realistic simulation environments, accounting for complex real-world factors such as robot kinodynamics and execution uncertainties, (2) SMART uses an execution monitor framework based on the Action Dependency Graph, facilitating seamless integration with various MAPF planners and robot models, and (3) SMART scales to thousands of robots. The code is publicly available at https://github.com/smart-mapf/smart with an online service available at https://smart-mapf.github.io/demo/.