# AeroAct: Action-Centered World-Action Models for Language-Conditioned Quadrotor Flight

arXiv: https://arxiv.org/abs/2607.14997 · HTML: https://mlsanity.com/paper/2607.14997/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in World models for navigation, one of 11 directions in World models.
- Builds on 4 of its 31 references that are in the map.
- First posted on arXiv 16 Jul 2026.

## Placed in

- World models / [World models for navigation](https://mlsanity.com/wm/world-models-for-navigation/)

## Builds on (most cited first, 4 of 4)

- [Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight](https://mlsanity.com/paper/2501.14377/) (2025)
- [DISCOVERSE: Efficient Robot Simulation in Complex High-Fidelity Environments](https://mlsanity.com/paper/2507.21981/) (2025)
- [MAD: Mapping-Aware World Models for Agile Quadrotor Flight](https://mlsanity.com/paper/2606.04534/) (2026)
- [GS-Playground: A High-Throughput Photorealistic Simulator for Vision-Informed Robot Learning](https://mlsanity.com/paper/2604.25459/) (2026)

## Abstract (arXiv)

Language-conditioned quadrotor flight requires a policy to ground semantic goals, anticipate the visual consequences of ego-motion, and output control references that remain smooth and dynamically executable under rapidly changing first-person views. Existing aerial vision-language navigation and vision-language-action methods commonly use discrete actions, high-level waypoints, or instantaneous velocity commands, which provide limited supervision about how flight actions change future observations. We present AeroAct, an action-centered world-action model (WAM) for quadrotor navigation. To the best of our knowledge, AeroAct is the first WAM instantiated and demonstrated for real-world aerial flight. The model adapts a pretrained video diffusion Transformer to predict local trajectory-action chunks from egocentric visual history, proprioception, and language. Future first-person frames are used during training as dense consequence supervision, while deployment directly decodes actions without generating future video. To obtain aligned visual, state, language, and dynamically feasible action data, we build a DiffAero-based pipeline with complementary Isaac Lab and 3D Gaussian splatting renderers. We further introduce a low-cost handheld collection device that couples camera observations with motion estimates to recreate flight-like egocentric trajectories, and a self-guidance procedure that improves temporal consistency across overlapping trajectory chunks. Closed-loop simulation and real-world experiments show that temporal visual context improves target tracking and object-search performance, and that WAM-based policies can be executed on a physical quadrotor.