# Learning Multi-Agent Loco-Manipulation for Long-Horizon Quadrupedal Pushing

arXiv: https://arxiv.org/abs/2411.07104 · HTML: https://mlsanity.com/paper/2411.07104/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 76 references that are in the map.
- First posted on arXiv 11 Nov 2024.

## Builds on (most cited first, 30 of 54)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Data-Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1805.08296/) (2018)
- [FeUdal Networks for Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1703.01161/) (2017)
- [HumanPlus: Humanoid Shadowing and Imitation from Humans](https://mlsanity.com/paper/2406.10454/) (2024)
- [Extreme Parkour with Legged Robots](https://mlsanity.com/paper/2309.14341/) (2023)
- [Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior](https://mlsanity.com/paper/2212.03238/) (2022)
- [Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion](https://mlsanity.com/paper/2210.10044/) (2022)
- [Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning](https://mlsanity.com/paper/2206.08686/) (2022)
- [Near-Optimal Representation Learning for Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1810.01257/) (2018)
- [Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks](https://mlsanity.com/paper/2110.03655/) (2021)
- [UMI on Legs: Making Manipulation Policies Mobile with Manipulation-Centric Whole-body Controllers](https://mlsanity.com/paper/2407.10353/) (2024)
- [Visual Whole-Body Control for Legged Loco-Manipulation](https://mlsanity.com/paper/2403.16967/) (2024)
- [WoCoCo: Learning Whole-Body Humanoid Control with Sequential Contacts](https://mlsanity.com/paper/2406.06005/) (2024)
- [RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control](https://mlsanity.com/paper/2012.03094/) (2020)
- [Fast and Efficient Locomotion via Learned Gait Transitions](https://mlsanity.com/paper/2104.04644/) (2021)
- [Articulated Object Interaction in Unknown Scenes with Whole-Body Mobile Manipulation](https://mlsanity.com/paper/2103.10534/) (2021)
- [Neural Volumetric Memory for Visual Locomotion Control](https://mlsanity.com/paper/2304.01201/) (2023)
- [Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Real](https://mlsanity.com/paper/1908.05224/) (2019)
- [DribbleBot: Dynamic Legged Manipulation in the Wild](https://mlsanity.com/paper/2304.01159/) (2023)
- [Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion](https://mlsanity.com/paper/2303.11330/) (2023)
- [Dynamic Handover: Throw and Catch with Bimanual Hands](https://mlsanity.com/paper/2309.05655/) (2023)
- [Feudal Multi-Agent Hierarchies for Cooperative Reinforcement Learning](https://mlsanity.com/paper/1901.08492/) (2019)
- [Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition](https://mlsanity.com/paper/2105.08692/) (2021)
- [Pedipulate: Enabling Manipulation Skills using a Quadruped Robot's Leg](https://mlsanity.com/paper/2402.10837/) (2024)
- [Learning Whole-body Manipulation for Quadrupedal Robot](https://mlsanity.com/paper/2308.16820/) (2023)
- [Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion](https://mlsanity.com/paper/2009.10019/) (2020)

## Built on by (most cited first, 13 of 13)

- [Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretraining](https://mlsanity.com/paper/2506.16475/) (2025)
- [OM2P: Offline Multi-Agent Mean-Flow Policy](https://mlsanity.com/paper/2508.06269/) (2025)
- [Decentralized Aerial Manipulation of a Cable-Suspended Load using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2508.01522/) (2025)
- [Closed-Loop Vision-Language Planning for Multi-Agent Coordination](https://mlsanity.com/paper/2502.10148/) (2025)
- [QuietPaw: Learning Quadrupedal Locomotion with Versatile Noise Preference Alignment](https://mlsanity.com/paper/2503.05035/) (2025)
- [CRAFT: Coaching Reinforcement Learning Autonomously using Foundation Models for Multi-Robot Coordination Tasks](https://mlsanity.com/paper/2509.14380/) (2025)
- [Efficient Navigation Among Movable Obstacles using a Mobile Manipulator via Hierarchical Policy Learning](https://mlsanity.com/paper/2506.15380/) (2025)
- [Robust and Safe Multi-Agent Reinforcement Learning with Communication for Autonomous Vehicles: From Simulation to Hardware](https://mlsanity.com/paper/2506.00982/) (2025)
- [ConPoSe: LLM-Guided Contact Point Selection for Scalable Cooperative Object Pushing](https://mlsanity.com/paper/2510.08705/) (2025)
- [Co-jump: Cooperative Jumping with Quadrupedal Robots via Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2602.10514/) (2026)
- [Dual-Quadruped Collaborative Transportation in Narrow Environments via Safe Reinforcement Learning](https://mlsanity.com/paper/2602.16353/) (2026)
- [Affordance-Based Hierarchical Reinforcement Learning for Quadruped Pedipulation](https://mlsanity.com/paper/2606.07506/) (2026)
- [Continual Quadruped Robots Coordination via Semantic Skill Discovery](https://mlsanity.com/paper/2606.08102/) (2026)

## Abstract (arXiv)

Recently, quadrupedal locomotion has achieved significant success, but their manipulation capabilities, particularly in handling large objects, remain limited, restricting their usefulness in demanding real-world applications such as search and rescue, construction, industrial automation, and room organization. This paper tackles the task of obstacle-aware, long-horizon pushing by multiple quadrupedal robots. We propose a hierarchical multi-agent reinforcement learning framework with three levels of control. The high-level controller integrates an RRT planner and a centralized adaptive policy to generate subgoals, while the mid-level controller uses a decentralized goal-conditioned policy to guide the robots toward these sub-goals. A pre-trained low-level locomotion policy executes the movement commands. We evaluate our method against several baselines in simulation, demonstrating significant improvements over baseline approaches, with 36.0% higher success rates and 24.5% reduction in completion time than the best baseline. Our framework successfully enables long-horizon, obstacle-aware manipulation tasks like Push-Cuboid and Push-T on Go1 robots in the real world.