# FlowPilot: Real-Time World-Action Modeling for Agile UAV Navigation

arXiv: https://arxiv.org/abs/2608.00635 · HTML: https://mlsanity.com/paper/2608.00635/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in World models for navigation, one of 11 directions in World models.
- Builds on 15 of its 32 references that are in the map.
- First posted on arXiv 1 Aug 2026.

## Placed in

- World models / [World models for navigation](https://mlsanity.com/wm/world-models-for-navigation/)

## Builds on (most cited first, 15 of 15)

- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [World Action Models are Zero-shot Policies](https://mlsanity.com/paper/2602.15922/) (2026)
- [Fast-WAM: Do World Action Models Need Test-time Future Imagination?](https://mlsanity.com/paper/2603.16666/) (2026)
- [Motus: A Unified Latent Action World Model](https://mlsanity.com/paper/2512.13030/) (2025)
- [NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration](https://mlsanity.com/paper/2310.07896/) (2023)
- [GigaWorld-Policy: An Efficient Action-Centered World--Action Model](https://mlsanity.com/paper/2603.17240/) (2026)
- [A General Optimization-based Framework for Local Odometry Estimation with Multiple Sensors](https://mlsanity.com/paper/1901.03638/) (2019)
- [Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight](https://mlsanity.com/paper/2501.14377/) (2025)
- [Aerial World Model for Long-horizon Visual Generation and Navigation in 3D Space](https://mlsanity.com/paper/2512.21887/) (2025)
- [Feedback Favors the Generalization of Neural ODEs](https://mlsanity.com/paper/2410.10253/) (2024)
- [MAD: Mapping-Aware World Models for Agile Quadrotor Flight](https://mlsanity.com/paper/2606.04534/) (2026)
- [DiffAero: A GPU-Accelerated Differentiable Simulation Framework for Efficient Quadrotor Policy Learning](https://mlsanity.com/paper/2509.10247/) (2025)
- [Optimizing Control-Friendly Trajectories with Self-Supervised Residual Learning](https://mlsanity.com/paper/2601.02738/) (2026)
- [Unified Meta-Representation and Feedback Calibration for General Disturbance Estimation](https://mlsanity.com/paper/2601.02762/) (2026)
- [Perception-Aware Time-Optimal Planning for Quadrotor Waypoint Flight](https://mlsanity.com/paper/2603.04305/) (2026)

## Built on by (most cited first, 2 of 2)

- [UAVs Meet Embodied Intelligence: Bridging Human Intents and Flying Dynamics Via Harnessing Physical-Digital AI Agents](https://mlsanity.com/paper/2609.18326/) (2026)
- [WOLF: World Model Guided LiDAR Exploration with Predictive Frontiers](https://mlsanity.com/paper/2609.23656/) (2026)

## Abstract (arXiv)

We present FlowPilot, a compact world-action model for real-time onboard UAV navigation from depth. Unlike map-then-optimize pipelines that require local reconstruction or end-to-end policies that lack explicit scene prediction, FlowPilot jointly denoises future depth observations and executable trajectories with flow matching. A dual-stream mixture-of-transformers couples video and action experts through shared attention, allowing future-scene prediction and trajectory generation to inform each other. At deployment, the model runs action-centrically and outputs only a trajectory. To ensure trackability, actions are parameterized as degree-7 Bernstein polynomials: the current state constrains the initial control points, and the network predicts five free control points, yielding C^2-continuous references with closed-form velocity, acceleration and jerk. FlowPilot is trained on a three-level depth pyramid spanning high-throughput simulation, photorealistic simulation, and real onboard data. In closed-loop simulation, it outperforms learning- and optimization-based baselines under increasing clutter and commanded speeds up to 8m/s. On a physical quadrotor, the full perception-to-action pipeline runs in under 18ms on a Jetson Orin NX and reaches 5.5m/s in cluttered indoor and forest environments using only onboard sensing and computation.