mlsanityData to 14 Oct 2026

Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning

arXiv 2310.10943 ↗First posted 17 Oct 2023cs.RO cs.LGcited by 111

Not on the map. Why?

Cited by 71 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

A central question in robotics is how to design a control system for an agile mobile robot. This paper studies this question systematically, focusing on a challenging setting: autonomous drone racing. We show that a neural network controller trained with reinforcement learning (RL) outperformed optimal control (OC) methods in this setting. We then investigated which fundamental factors have contributed to the success of RL or have limited OC. Our study indicates that the fundamental advantage of RL over OC is not that it optimizes its objective better but that it optimizes a better objective. OC decomposes the problem into planning and control with an explicit intermediate representation, such as a trajectory, that serves as an interface. This decomposition limits the range of behaviors that can be expressed by the controller, leading to inferior control performance when facing unmodeled effects. In contrast, RL can directly optimize a task-level objective and can leverage domain randomization to cope with model uncertainty, allowing the discovery of more robust control responses. Our findings allowed us to push an agile drone to its maximum performance, achieving a peak acceleration greater than 12 times the gravitational acceleration and a peak velocity of 108 kilometers per hour. Our policy achieved superhuman control within minutes of training on a standard workstation. This work presents a milestone in agile robotics and sheds light on the role of RL and OC in robot control.

Key facts. Cited by 111 papers in the map (data through 14 Oct 2026). Builds on 22 of its 53 references that are in the map. First posted on arXiv 17 Oct 2023.

Builds on

22 of 53 references.

References · 22

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  3. 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 382
  4. 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 254
  5. 2017Control of a Quadrotor with Reinforcement Learningcited by 92
  6. 2020Flightmare: A Flexible Quadrotor Simulatorcited by 84
  7. 2019Deep Drone Racing: From Simulation to Reality with Domain Randomizationcited by 76
  8. 2021Autonomous Drone Racing with Deep Reinforcement Learningcited by 60
  9. 2018Reinforcement Learning for UAV Attitude Controlcited by 48
  10. 2022A Benchmark Comparison of Learned Control Policies for Agile Quadrotor Flightcited by 47
  11. 2019Low Level Control of a Quadrotor with Deep Model-Based Reinforcement Learningcited by 44
  12. 2021NeuroBEM: Hybrid Aerodynamic Quadrotor Modelcited by 37
10 more
  1. 2020AlphaPilot: Autonomous Drone Racingcited by 30
  2. 2023Agilicious: Open-Source and Open-Hardware Agile Quadrotor for Vision-Based Flightcited by 30
  3. 2018PAMPC: Perception-Aware Model Predictive Control for Quadrotorscited by 28
  4. 2020HPIPM: a high-performance quadratic programming framework for model predictive controlcited by 23
  5. 2021Time-Optimal Planning for Quadrotor Waypoint Flightcited by 23
  6. 2021Model Predictive Contouring Control for Time-Optimal Quadrotor Flightcited by 21
  7. 2017Whole-Body Nonlinear Model Predictive Control Through Contacts for Quadrupedscited by 20
  8. 2021The Artificial Intelligence behind the winning entry to the 2019 AI Robotic Racing Competitioncited by 8
  9. 2021Air taxi service for urban mobility: A critical review of recent developments, future challenges, and opportunitiescited by 7
  10. 2017Minimum-time trajectory generation for quadrotors in constrained environmentscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 38; 2025: 41; 2026: 25.

111 papers.

Sort:

By area:

  1. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 218
  2. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  3. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 144
  4. 2024Agile But Safe: Learning Collision-Free High-Speed Legged Locomotioncited by 73
  5. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 59
  6. 2024Learning Quadruped Locomotion Using Differentiable Simulationcited by 32
  7. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 32
  8. 2024Bootstrapping Reinforcement Learning with Imitation for Vision-Based Agile Flightcited by 31
  9. 2024Demonstrating Agile Flight from Pixels without State Estimationcited by 31
  10. 2025Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classificationcited by 26
  11. 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 24
  12. 2024Multi-Task Reinforcement Learning for Quadrotorscited by 20
38 more
  1. 2024Learning Quadrotor Control From Visual Features Using Differentiable Simulationcited by 19
  2. 2025See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned Aerial Navigationcited by 18
  3. 2024Safe Reinforcement Learning on the Constraint Manifold: Theory and Applicationscited by 15
  4. 2025Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flightcited by 15
  5. 2024Learning Speed Adaptation for Flight in Cluttercited by 14
  6. 2024Reinforcement Learning for Collision-free Flight Exploiting Deep Collision Encodingcited by 13
  7. 2024MAVRL: Learn to Fly in Cluttered Environments with Varying Speedcited by 13
  8. 2024Residual Policy Learning for Perceptive Quadruped Control Using Differentiable Simulationcited by 13
  9. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Modelscited by 10
  10. 2025RAPTOR: A Foundation Policy for Quadrotor Controlcited by 8
  11. 2024VisFly: An Efficient and Versatile Simulator for Training Vision-based Flightcited by 7
  12. 2024Automatic Environment Shaping is the Next Frontier in RLcited by 7
  13. 2025Flying on Point Clouds with Reinforcement Learningcited by 7
  14. 2025One Net to Rule Them All: Domain Randomization in Quadcopter Racing Across Different Platformscited by 7
  15. 2023Adaptive Stochastic Nonlinear Model Predictive Control with Look-ahead Deep Reinforcement Learning for Autonomous Vehicle Motion Controlcited by 6
  16. 2024Dashing for the Golden Snitch: Multi-Drone Time-Optimal Motion Planning with Multi-Agent Reinforcement Learningcited by 6
  17. 2024Environment as Policy: Learning to Race in Unseen Trackscited by 6
  18. 2025RAPID: Robust and Agile Planner Using Inverse Reinforcement Learning for Vision-Based Drone Navigationcited by 6
  19. 2024Deep Reinforcement Learning in Autonomous Car Path Planning and Control: A Surveycited by 5
  20. 2025Improving Value Estimation Critically Enhances Vanilla Policy Gradientcited by 5
  21. 2025Reactive Aerobatic Flight via Reinforcement Learningcited by 5
  22. 2024CoVO-MPC: Theoretical Analysis of Sampling-based MPC and Optimal Covariance Designcited by 4
  23. 2023Risk-anticipatory autonomous driving strategies considering vehicles' weights, based on hierarchical deep reinforcement learningcited by 4
  24. 2024Neuromorphic Attitude Estimation and Controlcited by 4
  25. 2025Decentralized Aerial Manipulation of a Cable-Suspended Load using Multi-Agent Reinforcement Learningcited by 4
  26. 2025Agile Flight Emerges from Multi-Agent Competitive Racingcited by 4
  27. 2026Non-Uniform Noise-to-Signal Ratio in the REINFORCE Policy-Gradient Estimatorcited by 4
  28. 2024SigmaRL: A Sample-Efficient and Generalizable Multi-Agent Reinforcement Learning Framework for Motion Planningcited by 3
  29. 2024A Survey on Reinforcement Learning Applications in SLAMcited by 3
  30. 2025Benchmarking Model Predictive Control and Reinforcement Learning Based Control for Legged Robot Locomotion in MuJoCo Simulationcited by 3
  31. 2025TACO: General Acrobatic Flight Control via Target-and-Command-Oriented Reinforcement Learningcited by 3
  32. 2025Drive Fast, Learn Faster: On-Board RL for High Performance Autonomous Racingcited by 3
  33. 2025Depth Transfer: Learning to See Like a Simulator for Real-World Drone Navigationcited by 3
  34. 2025Learned Controllers for Agile Quadrotors in Pursuit-Evasion Gamescited by 3
  35. 2025Mastering Diverse, Unknown, and Cluttered Tracks for Robust Vision-Based Drone Racingcited by 3
  36. 2024Advancing Monocular Video-Based Gait Analysis Using Motion Imitation with Physics-Based Simulationcited by 2
  37. 2024Time-optimal Flight in Cluttered Environments via Safe Reinforcement Learningcited by 2
  38. 2024Data-Driven Stochastic Optimal Control in Reproducing Kernel Hilbert Spacescited by 2

50 most cited of 111 All 111 (JSON)