# Openfly: A comprehensive platform for aerial vision-language navigation

arXiv: https://arxiv.org/abs/2502.18041 · HTML: https://mlsanity.com/paper/2502.18041/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 67 references that are in the map.
- First posted on arXiv 25 Feb 2025.

## Builds on (most cited first, 30 of 43)

- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models](https://mlsanity.com/paper/2311.17043/) (2023)
- [Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2004.02857/) (2020)
- [Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding](https://mlsanity.com/paper/2010.07954/) (2020)
- [REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments](https://mlsanity.com/paper/1904.10151/) (2019)
- [VideoAgent: Long-form Video Understanding with Large Language Model as Agent](https://mlsanity.com/paper/2403.10517/) (2024)
- [Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation](https://mlsanity.com/paper/1811.10092/) (2018)
- [NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models](https://mlsanity.com/paper/2305.16986/) (2023)
- [Vision-and-Dialog Navigation](https://mlsanity.com/paper/1907.04957/) (2019)
- [Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments](https://mlsanity.com/paper/1811.12354/) (2018)
- [NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation](https://mlsanity.com/paper/2402.15852/) (2024)
- [A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets](https://mlsanity.com/paper/2406.12080/) (2024)
- [NaVILA: Legged Robot Vision-Language-Action Model for Navigation](https://mlsanity.com/paper/2412.04453/) (2024)
- [Self-Monitoring Navigation Agent via Auxiliary Progress Estimation](https://mlsanity.com/paper/1901.03035/) (2019)
- [Revisiting the "Video" in Video-Language Understanding](https://mlsanity.com/paper/2206.01720/) (2022)
- [InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environment](https://mlsanity.com/paper/2406.04882/) (2024)
- [Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigation](https://mlsanity.com/paper/2203.02764/) (2022)
- [Waypoint Models for Instruction-guided Navigation in Continuous Environments](https://mlsanity.com/paper/2110.02207/) (2021)
- [GridMM: Grid Memory Map for Vision-and-Language Navigation](https://mlsanity.com/paper/2307.12907/) (2023)
- [AerialVLN: Vision-and-Language Navigation for UAVs](https://mlsanity.com/paper/2308.06735/) (2023)
- [Topological Planning with Transformers for Vision-and-Language Navigation](https://mlsanity.com/paper/2012.05292/) (2020)
- [Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction](https://mlsanity.com/paper/1809.00786/) (2018)
- [NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Models](https://mlsanity.com/paper/2407.12366/) (2024)
- [Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodology](https://mlsanity.com/paper/2410.07087/) (2024)
- [Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2109.15207/) (2021)
- [Cross-modal Map Learning for Vision and Language Navigation](https://mlsanity.com/paper/2203.05137/) (2022)
- [Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigation](https://mlsanity.com/paper/2210.07506/) (2022)
- [MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigation](https://mlsanity.com/paper/2401.07314/) (2024)
- [Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation](https://mlsanity.com/paper/1903.02547/) (2019)

## Built on by (most cited first, 30 of 46)

- [FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving](https://mlsanity.com/paper/2505.17685/) (2025)
- [VLA-AN: An Efficient and Onboard Vision-Language-Action Framework for Aerial Navigation in Complex Environments](https://mlsanity.com/paper/2512.15258/) (2025)
- [LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integration](https://mlsanity.com/paper/2512.22010/) (2025)
- [iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Framework](https://mlsanity.com/paper/2605.03941/) (2026)
- [WorldVLN: Autoregressive World Action Model for Aerial Vision-Language Navigation](https://mlsanity.com/paper/2605.15964/) (2026)
- [AirSim360: A Panoramic Simulation Platform within Drone View](https://mlsanity.com/paper/2512.02009/) (2025)
- [Aerial World Model for Long-horizon Visual Generation and Navigation in 3D Space](https://mlsanity.com/paper/2512.21887/) (2025)
- [AerialVLA: A Vision-Language-Action Model for UAV Navigation via Minimalist End-to-End Control](https://mlsanity.com/paper/2603.14363/) (2026)
- [RAVEN: Resilient Aerial Navigation via Open-Set Semantic Memory and Behavior Adaptation](https://mlsanity.com/paper/2509.23563/) (2025)
- [AirNav: A Large-Scale UAV Vision-and-Language Navigation Dataset with Natural and Diverse Instructions](https://mlsanity.com/paper/2601.03707/) (2026)
- [UrbanNav: Learning Language-Guided Urban Navigation from Web-Scale Human Trajectories](https://mlsanity.com/paper/2512.09607/) (2025)
- [ImagineUAV: Aerial Vision-Language Navigation via World-Action Modeling and Kinodynamic Planning](https://mlsanity.com/paper/2606.01205/) (2026)
- [WorldFly: A World-Model-Based Vision-Language-Action Model for UAV Navigation](https://mlsanity.com/paper/2606.06147/) (2026)
- [Self in Space: Benchmarking Self-Awareness and Spatial Cognition in UAV Embodied Intelligence](https://mlsanity.com/paper/2607.12477/) (2026)
- [AERMANI-VLM: Structured Prompting and Reasoning for Aerial Manipulation with Vision Language Models](https://mlsanity.com/paper/2511.01472/) (2025)
- [Fly0: Persistent Metric Anchoring for Zero-Shot Aerial Vision-Language Navigation](https://mlsanity.com/paper/2602.15875/) (2026)
- [TranSimHub:A Unified Air-Ground Simulation Platform for Multi-Modal Perception and Decision-Making](https://mlsanity.com/paper/2510.15365/) (2025)
- [Are VLMs Lost Between Sky and Space? LinkS2Bench for UAV-Satellite Dynamic Cross-View Spatial Intelligence](https://mlsanity.com/paper/2604.02020/) (2026)
- [AerialMetric: Benchmarking and Adapting UAV Monocular Metric Depth Estimation in the Real World](https://mlsanity.com/paper/2606.29716/) (2026)
- [SoraNav: Adaptive UAV Task-Centric Navigation via Zeroshot VLM Reasoning](https://mlsanity.com/paper/2510.25191/) (2025)
- [HTNav: A Hybrid Navigation Framework with Tiered Structure for Urban Aerial Vision-and-Language Navigation](https://mlsanity.com/paper/2604.08883/) (2026)
- [ActiveFly-Bench: Aligning Embodied Question Answering with Vision-Language-Action for Aerial Embodied Perception](https://mlsanity.com/paper/2607.10180/) (2026)
- [Uncertainty-Aware World Model for Aerial Image-Goal Navigation](https://mlsanity.com/paper/2608.05597/) (2026)
- [Mitigating Error Accumulation in Continuous Navigation via Memory-Augmented Kalman Filtering](https://mlsanity.com/paper/2602.11183/) (2026)
- [FreeFly-Thinking : Aligning Chain-of-Thought Reasoning with Continuous UAV Navigation](https://mlsanity.com/paper/2603.07181/) (2026)
- [SpatialFly: Implicit 3D Prior-Guided Visual Reparameterization for Continuous UAV Vision-and-Language Navigation](https://mlsanity.com/paper/2603.21046/) (2026)
- [Towards Precise Intent-Aligned VLA Aerial Navigation via Expert-Guided GRPO](https://mlsanity.com/paper/2606.02313/) (2026)
- [Parse, Search, and Confirmation: Training-Free Aerial Vision-and-Dialog Navigation with Chain-of-Thought Reasoning and Structured Spatial Memory](https://mlsanity.com/paper/2607.11529/) (2026)
- [DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigation](https://mlsanity.com/paper/2608.12308/) (2026)
- [Air-Ground Collaborative Vision-and-Language Navigation via Shared Bird's-Eye Maps](https://mlsanity.com/paper/2609.03483/) (2026)

## Abstract (arXiv)

Vision-Language Navigation (VLN) aims to guide agents by leveraging language instructions and visual cues, playing a pivotal role in embodied AI. Indoor VLN has been extensively studied, whereas outdoor aerial VLN remains underexplored. The potential reason is that outdoor aerial view encompasses vast areas, making data collection more challenging, which results in a lack of benchmarks. To address this problem, we propose OpenFly, a platform comprising various rendering engines, a versatile toolchain, and a large-scale benchmark for aerial VLN. Firstly, we integrate diverse rendering engines and advanced techniques for environment simulation, including Unreal Engine, GTA V, Google Earth, and 3D Gaussian Splatting (3D GS). Particularly, 3D GS supports real-to-sim rendering, further enhancing the realism of our environments. Secondly, we develop a highly automated toolchain for aerial VLN data collection, streamlining point cloud acquisition, scene semantic segmentation, flight trajectory creation, and instruction generation. Thirdly, based on the toolchain, we construct a large-scale aerial VLN dataset with 100k trajectories, covering diverse heights and lengths across 18 scenes. Moreover, we propose OpenFly-Agent, a keyframe-aware VLN model emphasizing key observations during flight. For benchmarking, extensive experiments and analyses are conducted, evaluating several recent VLN methods and showcasing the superiority of our OpenFly platform and agent. The toolchain, dataset, and codes will be open-sourced.