mlsanityData to 14 Oct 2026

Learning Vision-Based Agile Gap Traversal: Differentiable Simulation with a Warm-Started Critic

arXiv 2609.30696 ↗First posted 25 Sep 2026cs.RO cs.SY eess.SYcited by 0

On the map: Deep reinforcement learning methods → RL for robot navigation and UAV control

Abstract (arXiv)

Traversing narrow gaps is challenging for autonomous quadrotors, especially when control commands come directly from high-dimensional visual observations. Existing end-to-end methods often rely on behavior cloning or full-rollout backpropagation through time (BPTT) via differentiable simulation, which can limit policy performance or incur high training costs. We propose a two-stage reinforcement learning framework for more efficient ego-centric visuomotor gap-traversal policy training, leveraging quasi-analytical policy gradients (QPG) via differentiable simulation and critic warm-starting. The framework utilizes QPG to avoid backpropagation through visual rendering, reducing computation and memory costs while improving sample efficiency. In the first stage, an expert actor and critic are trained using privileged observations, including gap geometry. Unlike prior gap-traversal approaches, our training utilizing QPG does not require resetting the agent along optimized reference trajectories. In the second stage, a visual policy is trained using binary gap masks from two ego-centric cameras and low-dimensional observations, while its privileged critic is warm-started from the first stage. This substantially improves training efficiency and traversal success compared with cold-starting the critic or using full-rollout BPTT. Our framework does not require retraining the expert actor when system parameters change, enabling more efficient generalization across drone platforms than state-of-the-art visual gap-traversal methods based on action supervision. The learned visual policy also generalizes to gaps with unseen shapes. Extensive real-world experiments further demonstrate robust gap traversal using binary masks rendered online. Beyond gap traversal, the proposed framework is generic and can be extended to other visuomotor robot learning tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods. Builds on 26 of its 34 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

26 of 34 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Learning Agile Gate Traversal via Analytical Optimal Policy Gradient29 Aug
  2. 2024Learning Quadrotor Control From Visual Features Using Differentiable Simulation21 Oct
  3. 2024Demonstrating Agile Flight from Pixels without State Estimation18 Jun

Other references · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  5. 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 238
  6. 2022Accelerated Policy Learning with Parallel Differentiable Simulationcited by 104
  7. 2024Back to Newton's Laws: Learning Vision-based Agile Flight via Differentiable Physicscited by 45
  8. 2024Learning Quadruped Locomotion Using Differentiable Simulationcited by 32
  9. 2022Training Efficient Controllers via Analytic Policy Gradientcited by 22
  10. 2023Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flightcited by 19
  11. 2017Search-based Motion Planning for Aggressive Flight in SE(3)cited by 15
  12. 2024Whole-Body Control Through Narrow Gaps From Pixels To Actioncited by 14
11 more
  1. 2024Seeing Through Pixel Motion: Learning Obstacle Avoidance from Optical Flow with One Cameracited by 13
  2. 2021Flying Through a Narrow Gap Using End-to-end Deep Reinforcement Learning Augmented with Curriculum Learning and Sim2Realcited by 10
  3. 2023Learning Agile Flights through Narrow Gaps with Varying Angles using Onboard Sensingcited by 8
  4. 2016Aggressive Quadrotor Flight through Narrow Gaps with Onboard Sensing and Computing using Active Visioncited by 7
  5. 2024OmniNxt: A Fully Open-source and Compact Aerial Robot with Omnidirectional Visual Perceptioncited by 6
  6. 2022Learning Agile Flight Maneuvers: Deep SE(3) Motion Planning and Control for Quadrotorscited by 5
  7. 2025Accelerating Visual-Policy Learning through Parallel Differentiable Simulationcited by 5
  8. 2019Flying through a narrow gap using neural network: an end-to-end planning and control approachcited by 4
  9. 2026Precise Aggressive Aerial Maneuvers with Sensorimotor Policiescited by 3
  10. 2026Vector Field Augmented Differentiable Policy Learning for Vision-Based Drone Racingcited by 2
  11. 2026Vision-Based End-to-End Learning for UAV Traversal of Irregular Gaps via Differentiable Simulationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Simple but Stable, Fast and Safe: Achieve End-to-end Control by High-Fidelity Differentiable Simulation
  2. 2024Bootstrapping Reinforcement Learning with Imitation for Vision-Based Agile Flight
  3. 2026VisFly-Lab: Unified Differentiable Framework for First-Order Reinforcement Learning of Quadrotor Control
  4. 2024VisFly: An Efficient and Versatile Simulator for Training Vision-based Flight
  5. 2022Learning Perception-Aware Agile Flight in Cluttered Environments
  6. 2026MAVEN: A Meta-Reinforcement Learning Framework for Varying-Dynamics Expertise in Agile Quadrotor Maneuvers
  7. 2025Quadrotor Navigation using Reinforcement Learning with Privileged Information
  8. 2025Mastering Diverse, Unknown, and Cluttered Tracks for Robust Vision-Based Drone Racing