# Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy Optimization

arXiv: https://arxiv.org/abs/1911.05478 · HTML: https://mlsanity.com/paper/1911.05478/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Nov 2019.

## Built on by (most cited first, 17 of 17)

- [A Survey on Reinforcement Learning in Aviation Applications](https://mlsanity.com/paper/2211.02147/) (2022)
- [Robust Control for Dynamical Systems With Non-Gaussian Noise via Formal Abstractions](https://mlsanity.com/paper/2301.01526/) (2023)
- [Sampling-Based Robust Control of Autonomous Systems with Non-Gaussian Noise](https://mlsanity.com/paper/2110.12662/) (2021)
- [Data-Efficient Deep Reinforcement Learning for Attitude Control of Fixed-Wing UAVs: Field Experiments](https://mlsanity.com/paper/2111.04153/) (2021)
- [Deep Reinforcement Learning for Trajectory Path Planning and Distributed Inference in Resource-Constrained UAV Swarms](https://mlsanity.com/paper/2212.11201/) (2022)
- [Soft Actor-Critic Deep Reinforcement Learning for Fault Tolerant Flight Control](https://mlsanity.com/paper/2202.09262/) (2022)
- [BVR Gym: A Reinforcement Learning Environment for Beyond-Visual-Range Air Combat](https://mlsanity.com/paper/2403.17533/) (2024)
- [Prediction-Free, Real-Time Flexible Control of Tidal Lagoons through Proximal Policy Optimisation: A Case Study for the Swansea Lagoon](https://mlsanity.com/paper/2106.10360/) (2021)
- [Algorithmic collusion in a two-sided market: A rideshare example](https://mlsanity.com/paper/2405.02835/) (2024)
- [Introduction to Reinforcement Learning](https://mlsanity.com/paper/2408.07712/) (2024)
- [Effective Reinforcement Learning Control using Conservative Soft Actor-Critic](https://mlsanity.com/paper/2505.03356/) (2025)
- [Adversarial Reinforcement Learning for Robust Control of Fixed-Wing Aircraft under Model Uncertainty](https://mlsanity.com/paper/2510.16650/) (2025)
- [Automatic Design Method of Building Pipeline Layout Based on Deep Reinforcement Learning](https://mlsanity.com/paper/2305.10760/) (2023)
- [Safety, Trust, and Ethics Considerations for Human-AI Teaming in Aerospace Control](https://mlsanity.com/paper/2311.08943/) (2023)
- [Scalable Multi-agent Reinforcement Learning for Factory-wide Dynamic Scheduling](https://mlsanity.com/paper/2409.13571/) (2024)
- [Model-Free versus Model-Based Reinforcement Learning for Fixed-Wing UAV Attitude Control Under Varying Wind Conditions](https://mlsanity.com/paper/2409.17896/) (2024)
- [FALCON-S: Fixed-wing ground-effect Aerodynamics Simulator and Flight Control Learning Suite](https://mlsanity.com/paper/2609.06046/) (2026)

## Abstract (arXiv)

Contemporary autopilot systems for unmanned aerial vehicles (UAVs) are far more limited in their flight envelope as compared to experienced human pilots, thereby restricting the conditions UAVs can operate in and the types of missions they can accomplish autonomously. This paper proposes a deep reinforcement learning (DRL) controller to handle the nonlinear attitude control problem, enabling extended flight envelopes for fixed-wing UAVs. A proof-of-concept controller using the proximal policy optimization (PPO) algorithm is developed, and is shown to be capable of stabilizing a fixed-wing UAV from a large set of initial conditions to reference roll, pitch and airspeed values. The training process is outlined and key factors for its progression rate are considered, with the most important factor found to be limiting the number of variables in the observation vector, and including values for several previous time steps for these variables. The trained reinforcement learning (RL) controller is compared to a proportional-integral-derivative (PID) controller, and is found to converge in more cases than the PID controller, with comparable performance. Furthermore, the RL controller is shown to generalize well to unseen disturbances in the form of wind and turbulence, even in severe disturbance conditions.