# Learning Agile Gate Traversal via Analytical Optimal Policy Gradient

arXiv: https://arxiv.org/abs/2508.21592 · HTML: https://mlsanity.com/paper/2508.21592/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 29 Aug 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 16 of 16)

- [Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning](https://mlsanity.com/paper/2310.10943/) (2023)
- [An Analysis of SVD for Deep Rotation Estimation](https://mlsanity.com/paper/2006.14616/) (2020)
- [acados: a modular open-source framework for fast embedded optimal control](https://mlsanity.com/paper/1910.13753/) (2019)
- [Learning with 3D rotations, a hitchhiker's guide to SO(3)](https://mlsanity.com/paper/2404.11735/) (2024)
- [Safe Pontryagin Differentiable Programming](https://mlsanity.com/paper/2105.14937/) (2021)
- [Model Predictive Contouring Control for Time-Optimal Quadrotor Flight](https://mlsanity.com/paper/2108.13205/) (2021)
- [Policy Search for Model Predictive Control with Application to Agile Drone Flight](https://mlsanity.com/paper/2112.03850/) (2021)
- [Geometrically Constrained Trajectory Optimization for Multicopters](https://mlsanity.com/paper/2103.00190/) (2021)
- [Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flight](https://mlsanity.com/paper/2306.09852/) (2023)
- [Search-based Motion Planning for Aggressive Flight in SE(3)](https://mlsanity.com/paper/1710.02748/) (2017)
- [Whole-Body Control Through Narrow Gaps From Pixels To Action](https://mlsanity.com/paper/2409.00895/) (2024)
- [Learning Agile Flights through Narrow Gaps with Varying Angles using Onboard Sensing](https://mlsanity.com/paper/2302.11233/) (2023)
- [Aggressive Quadrotor Flight through Narrow Gaps with Onboard Sensing and Computing using Active Vision](https://mlsanity.com/paper/1612.00291/) (2016)
- [One Net to Rule Them All: Domain Randomization in Quadcopter Racing Across Different Platforms](https://mlsanity.com/paper/2504.21586/) (2025)
- [Differentiable Collision Detection for a Set of Convex Primitives](https://mlsanity.com/paper/2207.00669/) (2022)
- [Learning Agile Flight Maneuvers: Deep SE(3) Motion Planning and Control for Quadrotors](https://mlsanity.com/paper/2209.11097/) (2022)

## Built on by (most cited first, 1 of 1)

- [Learning Vision-Based Agile Gap Traversal: Differentiable Simulation with a Warm-Started Critic](https://mlsanity.com/paper/2609.30696/) (2026)

## Abstract (arXiv)

Traversing narrow gates presents a significant challenge and has become a standard benchmark for evaluating agile and precise quadrotor flight. Traditional modularized autonomous flight stacks require extensive design and parameter tuning, while end-to-end reinforcement learning (RL) methods often suffer from low sample efficiency, limited interpretability, and degraded disturbance rejection under unseen perturbations. In this work, we present a novel hybrid framework that adaptively fine-tunes model predictive control (MPC) parameters online using outputs from a neural network (NN) trained offline. The NN jointly predicts a reference pose and cost function weights, conditioned on the coordinates of the gate corners and the current drone state. To achieve efficient training, we derive analytical policy gradients not only for the MPC module but also for an optimization-based gate traversal detection module. Hardware experiments demonstrate agile and accurate gate traversal with peak accelerations of 30 m/s2, as well as recovery within 0.85 s following body-rate disturbances exceeding 1146 deg/s.