# Decentralized Aerial Manipulation of a Cable-Suspended Load using Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2508.01522 · HTML: https://mlsanity.com/paper/2508.01522/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 51 references that are in the map.
- First posted on arXiv 2 Aug 2025.

## Builds on (most cited first, 30 of 30)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments](https://mlsanity.com/paper/2301.04195/) (2023)
- [Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning](https://mlsanity.com/paper/2310.10943/) (2023)
- [From Motor Control to Team Play in Simulated Humanoid Football](https://mlsanity.com/paper/2105.12196/) (2021)
- [Robust Autonomy Emerges from Self-Play](https://mlsanity.com/paper/2502.03349/) (2025)
- [skrl: Modular and Flexible Library for Reinforcement Learning](https://mlsanity.com/paper/2202.03825/) (2022)
- [A Benchmark Comparison of Learned Control Policies for Agile Quadrotor Flight](https://mlsanity.com/paper/2202.10796/) (2022)
- [CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamics](https://mlsanity.com/paper/2406.14558/) (2024)
- [OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control](https://mlsanity.com/paper/2309.12825/) (2023)
- [Learning to Fly in Seconds](https://mlsanity.com/paper/2311.13081/) (2023)
- [Bootstrapping Reinforcement Learning with Imitation for Vision-Based Agile Flight](https://mlsanity.com/paper/2403.12203/) (2024)
- [Agilicious: Open-Source and Open-Hardware Agile Quadrotor for Vision-Based Flight](https://mlsanity.com/paper/2307.06100/) (2023)
- [A Comparative Study of Nonlinear MPC and Differential-Flatness-Based Control for Quadrotor Agile Flight](https://mlsanity.com/paper/2109.01365/) (2021)
- [Decentralized Control of Quadrotor Swarms with End-to-end Deep Reinforcement Learning](https://mlsanity.com/paper/2109.07735/) (2021)
- [Accurate Tracking of Aggressive Quadrotor Trajectories using Incremental Nonlinear Dynamic Inversion and Differential Flatness](https://mlsanity.com/paper/1809.04048/) (2018)
- [Learning Multi-Agent Loco-Manipulation for Long-Horizon Quadrupedal Pushing](https://mlsanity.com/paper/2411.07104/) (2024)
- [MARLadona -- Towards Cooperative Team Play Using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2409.20326/) (2024)
- [Agile and Cooperative Aerial Manipulation of a Cable-Suspended Load](https://mlsanity.com/paper/2501.18802/) (2025)
- [Nonlinear Model Predictive Control for Cooperative Transportation and Manipulation of Cable Suspended Payloads with Multiple Quadrotors](https://mlsanity.com/paper/2303.06165/) (2023)
- [RotorTM: A Flexible Simulator for Aerial Transportation and Manipulation](https://mlsanity.com/paper/2205.05140/) (2022)
- [Proximal Policy Optimization for Tracking Control Exploiting Future Reference Information](https://mlsanity.com/paper/2107.09647/) (2021)
- [Auto-Multilift: Distributed Learning and Control for Cooperative Load Transportation With Quadrotors](https://mlsanity.com/paper/2406.04858/) (2024)
- [Decentralized Navigation of a Cable-Towed Load using Quadrupedal Robot Team via MARL](https://mlsanity.com/paper/2503.18221/) (2025)
- [Efficient Optimization-based Cable Force Allocation for Geometric Control of a Multirotor Team Transporting a Payload](https://mlsanity.com/paper/2304.02359/) (2023)

## Built on by (most cited first, 4 of 4)

- [DiffCoord: Differentiable Coordination for Distributed Multi-Agent Trajectory Optimization](https://mlsanity.com/paper/2509.01630/) (2025)
- [Autonomous Aerial Manipulation via Contextual Contrastive Meta Reinforcement Learning](https://mlsanity.com/paper/2606.08533/) (2026)
- [ASTER: Attitude-aware Suspended-payload Quadrotor Traversal via Efficient Reinforcement Learning](https://mlsanity.com/paper/2603.10715/) (2026)
- [Safe and Scalable Multi-Drone Payload Transport via CBF-based Reinforcement Learning with Zero-Shot Sim-to-Real Transfer](https://mlsanity.com/paper/2607.20665/) (2026)

## Abstract (arXiv)

This paper presents the first decentralized method to enable real-world 6-DoF manipulation of a cable-suspended load using a team of Micro-Aerial Vehicles (MAVs). Our method leverages multi-agent reinforcement learning (MARL) to train an outer-loop control policy for each MAV. Unlike state-of-the-art controllers that utilize a centralized scheme, our policy does not require global states, inter-MAV communications, nor neighboring MAV information. Instead, agents communicate implicitly through load pose observations alone, which enables high scalability and flexibility. It also significantly reduces computing costs during inference time, enabling onboard deployment of the policy. In addition, we introduce a new action space design for the MAVs using linear acceleration and body rates. This choice, combined with a robust low-level controller, enables reliable sim-to-real transfer despite significant uncertainties caused by cable tension during dynamic 3D motion. We validate our method in various real-world experiments, including full-pose control under load model uncertainties, showing setpoint tracking performance comparable to the state-of-the-art centralized method. We also demonstrate cooperation amongst agents with heterogeneous control policies, and robustness to the complete in-flight loss of one MAV. Videos of experiments: https://autonomousrobots.nl/paper_websites/aerial-manipulation-marl