# OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control

arXiv: https://arxiv.org/abs/2309.12825 · HTML: https://mlsanity.com/paper/2309.12825/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 47 references that are in the map.
- First posted on arXiv 22 Sep 2023.

## Builds on (most cited first, 30 of 33)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [RLBench: The Robot Learning Benchmark & Learning Environment](https://mlsanity.com/paper/1909.12271/) (2019)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.08839/) (2020)
- [Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments](https://mlsanity.com/paper/2301.04195/) (2023)
- [Trust Region Policy Optimisation in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2109.11251/) (2021)
- [Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning](https://mlsanity.com/paper/2206.08686/) (2022)
- [Control of a Quadrotor with Reinforcement Learning](https://mlsanity.com/paper/1707.05110/) (2017)
- [Flightmare: A Flexible Quadrotor Simulator](https://mlsanity.com/paper/2009.00563/) (2020)
- [Learning to Fly -- a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control](https://mlsanity.com/paper/2103.02142/) (2021)
- [Autonomous Drone Racing with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.08624/) (2021)
- [TorchRL: A data-driven decision-making library for PyTorch](https://mlsanity.com/paper/2306.00577/) (2023)
- [Reinforcement Learning for UAV Attitude Control](https://mlsanity.com/paper/1804.04154/) (2018)
- [A Benchmark Comparison of Learned Control Policies for Agile Quadrotor Flight](https://mlsanity.com/paper/2202.10796/) (2022)
- [Low Level Control of a Quadrotor with Deep Model-Based Reinforcement Learning](https://mlsanity.com/paper/1901.03737/) (2019)
- [Learning Deep Control Policies for Autonomous Aerial Vehicles with MPC-Guided Policy Search](https://mlsanity.com/paper/1509.06791/) (2015)
- [Deep Drone Racing: Learning Agile Flight in Dynamic Environments](https://mlsanity.com/paper/1806.08548/) (2018)
- [Model-Based Meta-Reinforcement Learning for Flight with Suspended Payloads](https://mlsanity.com/paper/2004.11345/) (2020)
- [FlightGoggles: A Modular Framework for Photorealistic Camera, Exteroceptive Sensor, and Dynamics Simulation](https://mlsanity.com/paper/1905.11377/) (2019)

## Built on by (most cited first, 30 of 33)

- [NavRL: Learning Safe Flight in Dynamic Environments](https://mlsanity.com/paper/2409.15634/) (2024)
- [SOUS VIDE: Cooking Visual Drone Navigation Policies in a Gaussian Splatting Vacuum](https://mlsanity.com/paper/2412.16346/) (2024)
- [ACEGEN: Reinforcement learning of generative chemical agents for drug discovery](https://mlsanity.com/paper/2405.04657/) (2024)
- [What Matters in Learning A Zero-Shot Sim-to-Real RL Policy for Quadrotor Control? A Comprehensive Study](https://mlsanity.com/paper/2412.11764/) (2024)
- [BEDI: A Comprehensive Benchmark for Evaluating Embodied Agents on UAVs](https://mlsanity.com/paper/2505.18229/) (2025)
- [MarineGym: A High-Performance Reinforcement Learning Platform for Underwater Robotics](https://mlsanity.com/paper/2503.09203/) (2025)
- [VisFly: An Efficient and Versatile Simulator for Training Vision-based Flight](https://mlsanity.com/paper/2407.14783/) (2024)
- [CARLA-Air: Fly Drones Inside a CARLA World -- A Unified Infrastructure for Air-Ground Embodied Intelligence](https://mlsanity.com/paper/2603.28032/) (2026)
- [Online Planning for Multi-UAV Pursuit-Evasion in Unknown Environments Using Deep Reinforcement Learning](https://mlsanity.com/paper/2409.15866/) (2024)
- [VolleyBots: A Testbed for Multi-Drone Volleyball Game Combining Motion Control and Strategic Play](https://mlsanity.com/paper/2502.01932/) (2025)
- [Decentralized Aerial Manipulation of a Cable-Suspended Load using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2508.01522/) (2025)
- [RoVerFly: Robust and Versatile Implicit Hybrid Control of Quadrotor-Payload Systems](https://mlsanity.com/paper/2509.11149/) (2025)
- [Learned Controllers for Agile Quadrotors in Pursuit-Evasion Games](https://mlsanity.com/paper/2506.02849/) (2025)
- [DiffAero: A GPU-Accelerated Differentiable Simulation Framework for Efficient Quadrotor Policy Learning](https://mlsanity.com/paper/2509.10247/) (2025)
- [AirDreamer: Generalist Drone Navigation with World Models](https://mlsanity.com/paper/2606.03252/) (2026)
- [Current applications and potential future directions of reinforcement learning-based Digital Twins in agriculture](https://mlsanity.com/paper/2406.08854/) (2024)
- [Flying Quadrotors in Tight Formations using Learning-based Model Predictive Control](https://mlsanity.com/paper/2410.09727/) (2024)
- [Neural Internal Model Control: Learning a Robust Control Policy via Predictive Error Feedback](https://mlsanity.com/paper/2411.13079/) (2024)
- [A Dual Curriculum Learning Framework for Multi-UAV Pursuit-Evasion in Diverse Environments](https://mlsanity.com/paper/2312.12255/) (2023)
- [Curriculum-based Sample Efficient Reinforcement Learning for Robust Stabilization of a Quadrotor](https://mlsanity.com/paper/2501.18490/) (2025)
- [Towards Task-Oriented Flying: Framework, Infrastructure, and Principles](https://mlsanity.com/paper/2504.15129/) (2025)
- [MoDeSuite: Robot Learning Task Suite for Benchmarking Mobile Manipulation with Deformable Objects](https://mlsanity.com/paper/2507.21796/) (2025)
- [JuggleRL: Mastering Ball Juggling with a Quadrotor via Deep Reinforcement Learning](https://mlsanity.com/paper/2509.24892/) (2025)
- [RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments](https://mlsanity.com/paper/2610.10409/) (2026)
- [MASP: Scalable GNN-based Planning for Multi-Agent Navigation](https://mlsanity.com/paper/2312.02522/) (2023)
- [Communication-Free Collective Navigation for a Swarm of UAVs via LiDAR-Based Deep Reinforcement Learning](https://mlsanity.com/paper/2601.13657/) (2026)
- [Conformal Reachability for Safe Control in Unknown Environments](https://mlsanity.com/paper/2602.03799/) (2026)
- [Decentralized End-to-End Multi-AAV Pursuit Using Predictive Spatio-Temporal Observation via Deep Reinforcement Learning](https://mlsanity.com/paper/2603.24238/) (2026)
- [PerchRL: Vision-Based Agile Perching on Inclined Platforms under Rapid and Irregular Motion](https://mlsanity.com/paper/2606.03441/) (2026)
- [MuJoCo-Drones-Gym: A GPU-Accelerated Multi-Drone Simulator for Control and Reinforcement Learning](https://mlsanity.com/paper/2606.08039/) (2026)

## Abstract (arXiv)

In this work, we introduce OmniDrones, an efficient and flexible platform tailored for reinforcement learning in drone control, built on Nvidia's Omniverse Isaac Sim. It employs a bottom-up design approach that allows users to easily design and experiment with various application scenarios on top of GPU-parallelized simulations. It also offers a range of benchmark tasks, presenting challenges ranging from single-drone hovering to over-actuated system tracking. In summary, we propose an open-sourced drone simulation platform, equipped with an extensive suite of tools for drone learning. It includes 4 drone models, 5 sensor modalities, 4 control modes, over 10 benchmark tasks, and a selection of widely used RL baselines. To showcase the capabilities of OmniDrones and to support future research, we also provide preliminary results on these benchmark tasks. We hope this platform will encourage further studies on applying RL to practical drone systems.