# ProxFly: Robust Control for Close Proximity Quadcopter Flight via Residual Reinforcement Learning

arXiv: https://arxiv.org/abs/2409.13193 · HTML: https://mlsanity.com/paper/2409.13193/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 21 of its 33 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Empirical Evaluation of Rectified Activations in Convolutional Network](https://mlsanity.com/paper/1505.00853/) (2015)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [Residual Policy Learning](https://mlsanity.com/paper/1812.06298/) (2018)
- [Neural Lander: Stable Drone Landing Control using Learned Dynamics](https://mlsanity.com/paper/1811.08027/) (2018)
- [Learning to Fly -- a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control](https://mlsanity.com/paper/2103.02142/) (2021)
- [Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulation](https://mlsanity.com/paper/1904.06703/) (2019)
- [Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2408.09675/) (2024)
- [Learning to Fly in Seconds](https://mlsanity.com/paper/2311.13081/) (2023)
- [Model-Based Meta-Reinforcement Learning for Flight with Suspended Payloads](https://mlsanity.com/paper/2004.11345/) (2020)
- [Learning a Single Near-hover Position Controller for Vastly Different Quadcopters](https://mlsanity.com/paper/2209.09232/) (2022)
- [Neural-Swarm: Decentralized Close-Proximity Multirotor Control Using Learned Interactions](https://mlsanity.com/paper/2003.02992/) (2020)
- [Neural-Swarm2: Planning and Control of Heterogeneous Multirotor Swarms using Learned Interactions](https://mlsanity.com/paper/2012.05457/) (2020)
- [End-to-end Reinforcement Learning for Time-Optimal Quadcopter Flight](https://mlsanity.com/paper/2311.16948/) (2023)
- [SO(2)-Equivariant Downwash Models for Close Proximity Flight](https://mlsanity.com/paper/2305.18983/) (2023)
- [Nonlinear MPC for Quadrotors in Close-Proximity Flight with Neural Network Downwash Prediction](https://mlsanity.com/paper/2304.07794/) (2023)
- [Flying batteries: In-flight battery switching to increase multirotor flight time](https://mlsanity.com/paper/1909.10091/) (2019)
- [Rectangular Pyramid Partitioning using Integrated Depth Sensors (RAPPIDS): A Fast Planner for Multicopter Navigation](https://mlsanity.com/paper/2003.01245/) (2020)
- [Downwash-aware Control Allocation for Over-actuated UAV Platforms](https://mlsanity.com/paper/2207.09645/) (2022)
- [Spreeze: High-Throughput Parallel Reinforcement Learning Framework](https://mlsanity.com/paper/2312.06126/) (2023)

## Built on by (most cited first, 4 of 4)

- [RAPTOR: A Foundation Policy for Quadrotor Control](https://mlsanity.com/paper/2509.11481/) (2025)
- [RoVerFly: Robust and Versatile Implicit Hybrid Control of Quadrotor-Payload Systems](https://mlsanity.com/paper/2509.11149/) (2025)
- [Efficient Real-World Autonomous Racing via Attenuated Residual Policy Optimization](https://mlsanity.com/paper/2603.12960/) (2026)
- [ResSafe: Learning Safety Filtering with Residual Reinforcement Learning for Humanoids](https://mlsanity.com/paper/2609.15988/) (2026)

## Abstract (arXiv)

This paper proposes the ProxFly, a residual deep Reinforcement Learning (RL)-based controller for close proximity quadcopter flight. Specifically, we design a residual module on top of a cascaded controller (denoted as basic controller) to generate high-level control commands, which compensate for external disturbances and thrust loss caused by downwash effects from other quadcopters. First, our method takes only the ego state and controllers' commands as inputs and does not rely on any communication between quadcopters, thereby reducing the bandwidth requirement. Through domain randomization, our method relaxes the requirement for accurate system identification and fine-tuned controller parameters, allowing it to adapt to changing system models. Meanwhile, our method not only reduces the proportion of unexplainable signals from the black box in control commands but also enables the RL training to skip the time-consuming exploration from scratch via guidance from the basic controller. We validate the effectiveness of the residual module in the simulation with different proximities. Moreover, we conduct the real close proximity flight test to compare ProxFly with the basic controller and an advanced model-based controller with complex aerodynamic compensation. Finally, we show that ProxFly can be used for challenging quadcopter mid-air docking, where two quadcopters fly in extreme proximity, and strong airflow significantly disrupts flight. However, our method can stabilize the quadcopter in this case and accomplish docking. The resources are available at https://github.com/ruiqizhang99/ProxFly.