# PyFlyt -- UAV Simulation Environments for Reinforcement Learning Research

arXiv: https://arxiv.org/abs/2304.01305 · HTML: https://mlsanity.com/paper/2304.01305/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2023.

## Built on by (most cited first, 9 of 9)

- [Gymnasium: A Standard Interface for Reinforcement Learning Environments](https://mlsanity.com/paper/2407.17032/) (2024)
- [MOMAland: A Set of Benchmarks for Multi-Objective Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2407.16312/) (2024)
- [Revisiting Experience Replayable Conditions](https://mlsanity.com/paper/2402.10374/) (2024)
- [From Reward Shaping to Q-Shaping: Achieving Unbiased Learning with LLM-Guided Knowledge](https://mlsanity.com/paper/2410.01458/) (2024)
- [World Models for Anomaly Detection during Model-Based Reinforcement Learning Inference](https://mlsanity.com/paper/2503.02552/) (2025)
- [DeLF: Designing Learning Environments with Foundation Models](https://mlsanity.com/paper/2401.08936/) (2024)
- [Effective Reinforcement Learning Control using Conservative Soft Actor-Critic](https://mlsanity.com/paper/2505.03356/) (2025)
- [Situationally-Aware Dynamics Learning](https://mlsanity.com/paper/2505.19574/) (2025)
- [Co-Design of Aeroelastic Systems with Deep Reinforcement Learning](https://mlsanity.com/paper/2607.15329/) (2026)

## Abstract (arXiv)

Unmanned aerial vehicles (UAVs) have numerous applications, but their efficient and optimal flight can be a challenge. Reinforcement Learning (RL) has emerged as a promising approach to address this challenge, yet there is no standardized library for testing and benchmarking RL algorithms on UAVs. In this paper, we introduce PyFlyt, a platform built on the Bullet physics engine with native Gymnasium API support. PyFlyt provides modular implementations of simple components, such as motors and lifting surfaces, allowing for the implementation of UAVs of arbitrary configurations. Additionally, PyFlyt includes various task definitions and multiple reward function settings for each vehicle type. We demonstrate the effectiveness of PyFlyt by training various RL agents for two UAV models: quadrotor and fixed-wing. Our findings highlight the effectiveness of RL in UAV control and planning, and further show that it is possible to train agents in sparse reward settings for UAVs. PyFlyt fills a gap in existing literature by providing a flexible and standardised platform for testing RL algorithms on UAVs. We believe that this will inspire more standardised research in this direction.