mlsanityData to 7 Oct 2026

Multi-Robot Collaboration through Reinforcement Learning and Abstract Simulation

arXiv 2503.05092 ↗First posted 7 Mar 2025cs.RO cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Teams of people coordinate to perform complex tasks by forming abstract mental models of world and agent dynamics. The use of abstract models contrasts with much recent work in robot learning that uses a high-fidelity simulator and reinforcement learning (RL) to obtain policies for physical robots. Motivated by this difference, we investigate the extent to which so-called abstract simulators can be used for multi-agent reinforcement learning (MARL) and the resulting policies successfully deployed on teams of physical robots. An abstract simulator models the robot's target task at a high-level of abstraction and discards many details of the world that could impact optimal decision-making. Policies are trained in an abstract simulator then transferred to the physical robot by making use of separately-obtained low-level perception and motion control modules. We identify three key categories of modifications to the abstract simulator that enable policy transfer to physical robots: simulation fidelity enhancements, training optimizations and simulation stochasticity. We then run an empirical study with extensive ablations to determine the value of each modification category for enabling policy transfer in cooperative robot soccer tasks. We also compare the performance of policies produced by our method with a well-tuned non-learning-based behavior architecture from the annual RoboCup competition and find that our approach leads to a similar level of performance. Broadly we show that MARL can be use to train cooperative physical robot behaviors using highly abstract models of the world.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 84 references that are in the map. First posted on arXiv 7 Mar 2025.

Builds on

41 of 84 references.

References · 41

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  4. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  5. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  6. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  7. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
  8. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  9. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 308
  10. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 277
  11. 2022Legged Locomotion in Challenging Terrains using Egocentric Visioncited by 131
  12. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 111
29 more
  1. 2019Entity Abstraction in Visual Model-Based Reinforcement Learningcited by 110
  2. 2021Learning High-Speed Flight in the Wildcited by 98
  3. 2020Model-Based Multi-Agent RL in Zero-Sum Markov Games with Near-Optimal Sample Complexitycited by 91
  4. 2017Control of a Quadrotor with Reinforcement Learningcited by 75
  5. 2018Benchmarking Reinforcement Learning Algorithms on Real-World Robotscited by 64
  6. 2021People construct simplified mental representations to plancited by 45
  7. 2020Sim2Real Transfer for Reinforcement Learning without Dynamics Randomizationcited by 43
  8. 2019Low Level Control of a Quadrotor with Deep Model-Based Reinforcement Learningcited by 40
  9. 2019Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Realcited by 39
  10. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  11. 2022Rethinking Sim2Real: Lower Fidelity Simulation Leads to Higher Sim2Real Transfer in Navigationcited by 28
  12. 2022Scalable Multi-Agent Model-Based Reinforcement Learningcited by 24
  13. 2024Back to Newton's Laws: Learning Vision-based Agile Flight via Differentiable Physicscited by 24
  14. 2020Learning to Fly via Deep Model-Based Reinforcement Learningcited by 19
  15. 2024Demonstrating Agile Flight from Pixels without State Estimationcited by 18
  16. 2022Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Realcited by 16
  17. 2020Learning Navigation Skills for Legged Robots with Learned Robot Embeddingscited by 14
  18. 2022Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computationcited by 14
  19. 2024Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learningcited by 14
  20. 2021A Framework for Real-World Multi-Robot Systems Running Decentralized GNN-Based Policiescited by 11
  21. 2020SuperSuit: Simple Microwrappers for Reinforcement Learning Environmentscited by 10
  22. 2023How to Learn and Generalize From Three Minutes of Data: Physics-Constrained and Uncertainty-Aware Neural Stochastic Differential Equationscited by 10
  23. 2022Multi-fidelity reinforcement learning framework for shape optimizationcited by 8
  24. 2021Model based Multi-agent Reinforcement Learning with Tensor Decompositionscited by 7
  25. 2020Towards Closing the Sim-to-Real Gap in Collaborative Multi-Robot Deep Reinforcement Learningcited by 6
  26. 2024Reinforcement Learning Within the Classical Robotics Stack: A Case Study in Robot Soccercited by 5
  27. 2019On Policy Learning Robust to Irreversible Events: An Application to Robotic In-Hand Manipulationcited by 4
  28. 2022Black-Box Safety Validation of Autonomous Systems: A Multi-Fidelity Reinforcement Learning Approachcited by 3
  29. 2020Trade-off on Sim2Real Learning: Real-world Learning Faster than Simulationscited by 1

Built on by

Not cited yet.