# Reinforcement Learning for UAV Attitude Control

arXiv: https://arxiv.org/abs/1804.04154 · HTML: https://mlsanity.com/paper/1804.04154/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Apr 2018.

## Built on by (most cited first, 30 of 48)

- [Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning](https://mlsanity.com/paper/2310.10943/) (2023)
- [Learning to Fly -- a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control](https://mlsanity.com/paper/2103.02142/) (2021)
- [OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control](https://mlsanity.com/paper/2309.12825/) (2023)
- [Demonstrating Agile Flight from Pixels without State Estimation](https://mlsanity.com/paper/2406.12505/) (2024)
- [Machine Learning-Aided Operations and Communications of Unmanned Aerial Vehicles: A Contemporary Survey](https://mlsanity.com/paper/2211.04324/) (2022)
- [Learning a Single Near-hover Position Controller for Vastly Different Quadcopters](https://mlsanity.com/paper/2209.09232/) (2022)
- [A Survey on Reinforcement Learning in Aviation Applications](https://mlsanity.com/paper/2211.02147/) (2022)
- [Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight](https://mlsanity.com/paper/2501.14377/) (2025)
- [UAVs Meet Agentic AI: A Multidomain Survey of Autonomous Aerial Intelligence and Agentic UAVs](https://mlsanity.com/paper/2506.08045/) (2025)
- [Joint Differentiable Optimization and Verification for Certified Reinforcement Learning](https://mlsanity.com/paper/2201.12243/) (2022)
- [Constrained Reinforcement Learning for Robotics via Scenario-Based Programming](https://mlsanity.com/paper/2206.09603/) (2022)
- [Video2Reward: Generating Reward Function from Videos for Legged Robot Behavior Learning](https://mlsanity.com/paper/2412.05515/) (2024)
- [Inclined Quadrotor Landing using Deep Reinforcement Learning](https://mlsanity.com/paper/2103.09043/) (2021)
- [Data-Efficient Deep Reinforcement Learning for Attitude Control of Fixed-Wing UAVs: Field Experiments](https://mlsanity.com/paper/2111.04153/) (2021)
- [Autonomous Unmanned Aerial Vehicle Navigation using Reinforcement Learning: A Systematic Review](https://mlsanity.com/paper/2208.12328/) (2022)
- [Minimizing Human Assistance: Augmenting a Single Demonstration for Deep Reinforcement Learning](https://mlsanity.com/paper/2209.11275/) (2022)
- [QuadSwarm: A Modular Multi-Quadrotor Simulator for Deep Reinforcement Learning with Direct Thrust Control](https://mlsanity.com/paper/2306.09537/) (2023)
- [Robust Deep Reinforcement Learning for Quadcopter Control](https://mlsanity.com/paper/2111.03915/) (2021)
- [Deep Reinforcement Learning for Sim-to-Real Policy Transfer of VTOL-UAVs Offshore Docking Operations](https://mlsanity.com/paper/2406.00887/) (2024)
- [Reinforcement Learning with Formal Performance Metrics for Quadcopter Attitude Control under Non-nominal Contexts](https://mlsanity.com/paper/2107.12942/) (2021)
- [Soft Actor-Critic Deep Reinforcement Learning for Fault Tolerant Flight Control](https://mlsanity.com/paper/2202.09262/) (2022)
- [Reliable and Efficient Data Collection in UAV-based IoT Networks](https://mlsanity.com/paper/2311.05303/) (2023)
- [BVR Gym: A Reinforcement Learning Environment for Beyond-Visual-Range Air Combat](https://mlsanity.com/paper/2403.17533/) (2024)
- [Reinforcement Learning with Imperfect Transition Predictions: A Bellman-Jensen Approach](https://mlsanity.com/paper/2510.18687/) (2025)
- [Drone swarm patrolling with uneven coverage requirements](https://mlsanity.com/paper/2107.00362/) (2021)
- [Adaptive DNN Surgery for Selfish Inference Acceleration with On-demand Edge Resource](https://mlsanity.com/paper/2306.12185/) (2023)
- [On the Design of Safe Continual RL Methods for Control of Nonlinear Systems](https://mlsanity.com/paper/2502.15922/) (2025)
- [Improving a Proportional Integral Controller with Reinforcement Learning on a Throttle Valve Benchmark](https://mlsanity.com/paper/2402.13654/) (2024)
- [Automated System-level Testing of Unmanned Aerial Systems](https://mlsanity.com/paper/2403.15857/) (2024)
- [Curriculum-based Sample Efficient Reinforcement Learning for Robust Stabilization of a Quadrotor](https://mlsanity.com/paper/2501.18490/) (2025)

## Abstract (arXiv)

Autopilot systems are typically composed of an "inner loop" providing stability and control, while an "outer loop" is responsible for mission-level objectives, e.g. way-point navigation. Autopilot systems for UAVs are predominately implemented using Proportional, Integral Derivative (PID) control systems, which have demonstrated exceptional performance in stable environments. However more sophisticated control is required to operate in unpredictable, and harsh environments. Intelligent flight control systems is an active area of research addressing limitations of PID control most recently through the use of reinforcement learning (RL) which has had success in other applications such as robotics. However previous work has focused primarily on using RL at the mission-level controller. In this work, we investigate the performance and accuracy of the inner control loop providing attitude control when using intelligent flight control systems trained with the state-of-the-art RL algorithms, Deep Deterministic Gradient Policy (DDGP), Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO). To investigate these unknowns we first developed an open-source high-fidelity simulation environment to train a flight controller attitude control of a quadrotor through RL. We then use our environment to compare their performance to that of a PID controller to identify if using RL is appropriate in high-precision, time-critical flight control.