# Robust Deep Reinforcement Learning for Quadcopter Control

arXiv: https://arxiv.org/abs/2111.03915 · HTML: https://mlsanity.com/paper/2111.03915/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 6 Nov 2021.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [Action Robust Reinforcement Learning and Applications in Continuous Control](https://mlsanity.com/paper/1901.09184/) (2019)
- [Control of a Quadrotor with Reinforcement Learning](https://mlsanity.com/paper/1707.05110/) (2017)
- [Deep Drone Racing: From Simulation to Reality with Domain Randomization](https://mlsanity.com/paper/1905.09727/) (2019)
- [Reinforcement Learning for UAV Attitude Control](https://mlsanity.com/paper/1804.04154/) (2018)
- [Sim-to-(Multi)-Real: Transfer of Low-Level Robust Control Policies to Multiple Quadrotors](https://mlsanity.com/paper/1903.04628/) (2019)
- [Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations](https://mlsanity.com/paper/1909.06993/) (2019)
- [Developmental Reinforcement Learning of Control Policy of a Quadcopter UAV with Thrust Vectoring Rotors](https://mlsanity.com/paper/2007.07793/) (2020)
- [Flight Control of Sliding Arm Quadcopter with Dynamic Structural Parameters](https://mlsanity.com/paper/2004.12920/) (2020)
- [Quaternion Feedback Based Autonomous Control of a Quadcopter UAV with Thrust Vectoring Rotors](https://mlsanity.com/paper/2006.15686/) (2020)

## Built on by (most cited first, 4 of 4)

- [Robust Deterministic Policy Gradient for Disturbance Attenuation and Its Application to Quadrotor Control](https://mlsanity.com/paper/2502.21057/) (2025)
- [Training Verifiably Robust Agents Using Set-Based Reinforcement Learning](https://mlsanity.com/paper/2408.09112/) (2024)
- [Tackling Snow-Induced Challenges: Safe Autonomous Lane-Keeping with Robust Reinforcement Learning](https://mlsanity.com/paper/2512.12987/) (2025)
- [Taming the Adversary: A Cost-to-Disturbance Ratio Approach to Adversarial Reinforcement Learning](https://mlsanity.com/paper/2603.12110/) (2026)

## Abstract (arXiv)

Deep reinforcement learning (RL) has made it possible to solve complex robotics problems using neural networks as function approximators. However, the policies trained on stationary environments suffer in terms of generalization when transferred from one environment to another. In this work, we use Robust Markov Decision Processes (RMDP) to train the drone control policy, which combines ideas from Robust Control and RL. It opts for pessimistic optimization to handle potential gaps between policy transfer from one environment to another. The trained control policy is tested on the task of quadcopter positional control. RL agents were trained in a MuJoCo simulator. During testing, different environment parameters (unseen during the training) were used to validate the robustness of the trained policy for transfer from one environment to another. The robust policy outperformed the standard agents in these environments, suggesting that the added robustness increases generality and can adapt to non-stationary environments. Codes: https://github.com/adipandas/gym_multirotor