# FlightDiffusion: Revolutionising Autonomous Drone Training with Diffusion Models Generating FPV Video

arXiv: https://arxiv.org/abs/2509.14082 · HTML: https://mlsanity.com/paper/2509.14082/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 18 of its 37 references that are in the map.
- First posted on arXiv 17 Sep 2025.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)

## Builds on (most cited first, 18 of 18)

- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [AerialVLN: Vision-and-Language Navigation for UAVs](https://mlsanity.com/paper/2308.06735/) (2023)
- [Embodied Navigation Foundation Model](https://mlsanity.com/paper/2509.12129/) (2025)
- [UAV-VLA: Vision-Language-Action System for Large Scale Aerial Mission Generation](https://mlsanity.com/paper/2501.05014/) (2025)
- [RaceVLA: VLA-based Racing Drone Navigation with Human-like Behaviour](https://mlsanity.com/paper/2503.02572/) (2025)
- [CGD: Constraint-Guided Diffusion Policies for UAV Trajectory Planning](https://mlsanity.com/paper/2405.01758/) (2024)
- [PRESTO: Fast Motion Planning Using Diffusion Models Based on Key-Configuration Environment Representation](https://mlsanity.com/paper/2409.16012/) (2024)
- [DroneDiffusion: Robust Quadrotor Dynamics Learning with Diffusion Models](https://mlsanity.com/paper/2409.11292/) (2024)
- [VLM-RRT: Vision Language Model Guided RRT Search for Autonomous UAV Navigation](https://mlsanity.com/paper/2505.23267/) (2025)
- [The Effect of Color Space Selection on Detectability and Discriminability of Colored Objects](https://mlsanity.com/paper/1702.05421/) (2017)
- [FM-Planner: Foundation Model Guided Path Planning for Autonomous Drone Navigation](https://mlsanity.com/paper/2505.20783/) (2025)
- [Automatic Generation of Aerobatic Flight in Complex Environments via Diffusion Models](https://mlsanity.com/paper/2504.15138/) (2025)
- [Contextualized Autonomous Drone Navigation using LLMs Deployed in Edge-Cloud Computing](https://mlsanity.com/paper/2504.00607/) (2025)
- [Improving Drone Racing Performance Through Iterative Learning MPC](https://mlsanity.com/paper/2508.01103/) (2025)
- [Vision-based Navigation of Unmanned Aerial Vehicles in Orchards: An Imitation Learning Approach](https://mlsanity.com/paper/2508.02617/) (2025)
- [Autonomous UAV Flight Navigation in Confined Spaces: A Reinforcement Learning Approach](https://mlsanity.com/paper/2508.16807/) (2025)
- [Synthetic vs. Real Training Data for Visual Navigation](https://mlsanity.com/paper/2509.11791/) (2025)

## Abstract (arXiv)

We present FlightDiffusion, a diffusion-model-based framework for training autonomous drones from first-person view (FPV) video. Our model generates realistic video sequences from a single frame, enriched with corresponding action spaces to enable reasoning-driven navigation in dynamic environments. Beyond direct policy learning, FlightDiffusion leverages its generative capabilities to synthesize diverse FPV trajectories and state-action pairs, facilitating the creation of large-scale training datasets without the high cost of real-world data collection. Our evaluation demonstrates that the generated trajectories are physically plausible and executable, with a mean position error of 0.25 m (RMSE 0.28 m) and a mean orientation error of 0.19 rad (RMSE 0.24 rad). This approach enables improved policy learning and dataset scalability, leading to superior performance in downstream navigation tasks. Results in simulated environments highlight enhanced robustness, smoother trajectory planning, and adaptability to unseen conditions. An ANOVA revealed no statistically significant difference between performance in simulation and reality (F(1, 16) = 0.394, p = 0.541), with success rates of M = 0.628 (SD = 0.162) and M = 0.617 (SD = 0.177), respectively, indicating strong sim-to-real transfer. The generated datasets provide a valuable resource for future UAV research. This work introduces diffusion-based reasoning as a promising paradigm for unifying navigation, action generation, and data synthesis in aerial robotics.