# Learning to Initialize Trajectory Optimization for Vision-Based Autonomous Flight in Unknown Environments

arXiv: https://arxiv.org/abs/2309.10683 · HTML: https://mlsanity.com/paper/2309.10683/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 19 Sep 2023.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems](https://mlsanity.com/paper/2003.04919/) (2020)
- [Learning High-Speed Flight in the Wild](https://mlsanity.com/paper/2110.05113/) (2021)
- [Learning Perception-Aware Agile Flight in Cluttered Environments](https://mlsanity.com/paper/2210.01841/) (2022)
- [MADER: Trajectory Planner in Multi-Agent and Dynamic Environments](https://mlsanity.com/paper/2010.11061/) (2020)
- [Neural-Swarm2: Planning and Control of Heterogeneous Multirotor Swarms using Learned Interactions](https://mlsanity.com/paper/2012.05457/) (2020)
- [Geometrically Constrained Trajectory Optimization for Multicopters](https://mlsanity.com/paper/2103.00190/) (2021)
- [Deep-PANTHER: Learning-Based Perception-Aware Trajectory Planner in Dynamic Environments](https://mlsanity.com/paper/2209.01268/) (2022)
- [PANTHER: Perception-Aware Trajectory Planner in Dynamic Environments](https://mlsanity.com/paper/2103.06372/) (2021)
- [Distributed Swarm Trajectory Optimization for Formation Flight in Dense Environments](https://mlsanity.com/paper/2109.07682/) (2021)

## Abstract (arXiv)

Autonomous flight in unknown environments requires precise spatial and temporal trajectory planning, often involving computationally expensive nonconvex optimization prone to local optima. To overcome these challenges, we present the Neural-Enhanced Trajectory Planner (NEO-Planner), a novel approach that leverages a Neural Network (NN) Planner to provide informed initial values for trajectory optimization. The NN-Planner is trained on a dataset generated by an expert planner using batch sampling, capturing multimodal trajectory solutions. It learns to predict spatial and temporal parameters for trajectories directly from raw sensor observations. NEO-Planner starts optimization from these predictions, accelerating computation speed while maintaining explainability. Furthermore, we introduce a robust online replanning framework that accommodates planning latency for smooth trajectory tracking. Extensive simulations demonstrate that NEO-Planner reduces optimization iterations by 20%, leading to a 26% decrease in computation time compared with pure optimization-based methods. It maintains trajectory quality comparable to baseline approaches and generalizes well to unseen environments. Real-world experiments validate its effectiveness for autonomous drone navigation in cluttered, unknown environments.