mlsanityData to 14 Oct 2026

Openfly: A comprehensive platform for aerial vision-language navigation

arXiv 2502.18041 ↗First posted 25 Feb 2025cs.CV cs.ROcited by 46

Not on the map. Why?

Abstract (arXiv)

Vision-Language Navigation (VLN) aims to guide agents by leveraging language instructions and visual cues, playing a pivotal role in embodied AI. Indoor VLN has been extensively studied, whereas outdoor aerial VLN remains underexplored. The potential reason is that outdoor aerial view encompasses vast areas, making data collection more challenging, which results in a lack of benchmarks. To address this problem, we propose OpenFly, a platform comprising various rendering engines, a versatile toolchain, and a large-scale benchmark for aerial VLN. Firstly, we integrate diverse rendering engines and advanced techniques for environment simulation, including Unreal Engine, GTA V, Google Earth, and 3D Gaussian Splatting (3D GS). Particularly, 3D GS supports real-to-sim rendering, further enhancing the realism of our environments. Secondly, we develop a highly automated toolchain for aerial VLN data collection, streamlining point cloud acquisition, scene semantic segmentation, flight trajectory creation, and instruction generation. Thirdly, based on the toolchain, we construct a large-scale aerial VLN dataset with 100k trajectories, covering diverse heights and lengths across 18 scenes. Moreover, we propose OpenFly-Agent, a keyframe-aware VLN model emphasizing key observations during flight. For benchmarking, extensive experiments and analyses are conducted, evaluating several recent VLN methods and showcasing the superiority of our OpenFly platform and agent. The toolchain, dataset, and codes will be open-sourced.

Key facts. Cited by 46 papers in the map (data through 14 Oct 2026). Builds on 43 of its 67 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

43 of 67 references.

References · 43

  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  2. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  3. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 546
  4. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  5. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  6. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 333
  7. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 332
  8. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
  9. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
  10. 2019Vision-and-Dialog Navigationcited by 250
  11. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 239
  12. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
31 more
  1. 2024A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasetscited by 216
  2. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 211
  3. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 176
  4. 2022Revisiting the "Video" in Video-Language Understandingcited by 169
  5. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 157
  6. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 132
  7. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 129
  8. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
  9. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 121
  10. 2020Topological Planning with Transformers for Vision-and-Language Navigationcited by 116
  11. 2018Mapping Instructions to Actions in 3D Environments with Visual Goal Predictioncited by 101
  12. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 98
  13. 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 95
  14. 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 94
  15. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  16. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  17. 2024MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigationcited by 86
  18. 2019Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigationcited by 81
  19. 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 67
  20. 2020EGO-Planner: An ESDF-free Gradient-based Local Planner for Quadrotorscited by 55
  21. 2022SensatUrban: Learning Semantics from Urban-Scale Photogrammetric Point Cloudscited by 50
  22. 2022Aerial Vision-and-Dialog Navigationcited by 49
  23. 2021SASRA: Semantically-aware Spatio-temporal Reasoning Agent for Vision-and-Language Navigation in Continuous Environmentscited by 48
  24. 2019Robust and Efficient Quadrotor Trajectory Generation for Fast Autonomous Flightcited by 41
  25. 2024Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigationcited by 39
  26. 2024CogNav: Cognitive Process Modeling for Object Goal Navigation with LLMscited by 33
  27. 2018PAMPC: Perception-Aware Model Predictive Control for Quadrotorscited by 28
  28. 2019Counterfactual Vision-and-Language Navigation via Adversarial Path Samplingcited by 22
  29. 2021Geometrically Constrained Trajectory Optimization for Multicopterscited by 19
  30. 2023CityRefer: Geography-aware 3D Visual Grounding Dataset on City-scale Point Cloud Datacited by 18
  31. 2024Open-Vocabulary Octree-Graph for 3D Scene Understandingcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 36.

46 papers.

Sort:

By area:

  1. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 166
  2. 2025VLA-AN: An Efficient and Onboard Vision-Language-Action Framework for Aerial Navigation in Complex Environmentscited by 23
  3. 2025LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integrationcited by 14
  4. 2026iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Frameworkcited by 14
  5. 2026WorldVLN: Autoregressive World Action Model for Aerial Vision-Language Navigationcited by 14
  6. 2025AirSim360: A Panoramic Simulation Platform within Drone Viewcited by 12
  7. 2025Aerial World Model for Long-horizon Visual Generation and Navigation in 3D Spacecited by 12
  8. 2026AerialVLA: A Vision-Language-Action Model for UAV Navigation via Minimalist End-to-End Controlcited by 11
  9. 2025RAVEN: Resilient Aerial Navigation via Open-Set Semantic Memory and Behavior Adaptationcited by 10
  10. 2026AirNav: A Large-Scale UAV Vision-and-Language Navigation Dataset with Natural and Diverse Instructionscited by 7
  11. 2025UrbanNav: Learning Language-Guided Urban Navigation from Web-Scale Human Trajectoriescited by 6
  12. 2026ImagineUAV: Aerial Vision-Language Navigation via World-Action Modeling and Kinodynamic Planningcited by 6
34 more
  1. 2026WorldFly: A World-Model-Based Vision-Language-Action Model for UAV Navigationcited by 6
  2. 2026Self in Space: Benchmarking Self-Awareness and Spatial Cognition in UAV Embodied Intelligencecited by 6
  3. 2025AERMANI-VLM: Structured Prompting and Reasoning for Aerial Manipulation with Vision Language Modelscited by 4
  4. 2026Fly0: Persistent Metric Anchoring for Zero-Shot Aerial Vision-Language Navigationcited by 4
  5. 2025TranSimHub:A Unified Air-Ground Simulation Platform for Multi-Modal Perception and Decision-Makingcited by 3
  6. 2026Are VLMs Lost Between Sky and Space? LinkS2Bench for UAV-Satellite Dynamic Cross-View Spatial Intelligencecited by 3
  7. 2026AerialMetric: Benchmarking and Adapting UAV Monocular Metric Depth Estimation in the Real Worldcited by 3
  8. 2025SoraNav: Adaptive UAV Task-Centric Navigation via Zeroshot VLM Reasoningcited by 2
  9. 2026HTNav: A Hybrid Navigation Framework with Tiered Structure for Urban Aerial Vision-and-Language Navigationcited by 2
  10. 2026ActiveFly-Bench: Aligning Embodied Question Answering with Vision-Language-Action for Aerial Embodied Perceptioncited by 2
  11. 2026Uncertainty-Aware World Model for Aerial Image-Goal Navigationcited by 2
  12. 2026Mitigating Error Accumulation in Continuous Navigation via Memory-Augmented Kalman Filteringcited by 1
  13. 2026FreeFly-Thinking : Aligning Chain-of-Thought Reasoning with Continuous UAV Navigationcited by 1
  14. 2026SpatialFly: Implicit 3D Prior-Guided Visual Reparameterization for Continuous UAV Vision-and-Language Navigationcited by 1
  15. 2026Towards Precise Intent-Aligned VLA Aerial Navigation via Expert-Guided GRPOcited by 1
  16. 2026Parse, Search, and Confirmation: Training-Free Aerial Vision-and-Dialog Navigation with Chain-of-Thought Reasoning and Structured Spatial Memorycited by 1
  17. 2026DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigationcited by 1
  18. 2026Air-Ground Collaborative Vision-and-Language Navigation via Shared Bird's-Eye Mapscited by 1
  19. 2026NavGen: Visual Generative Models as a Scalable Data Engine for Embodied 3D Navigationcited by 1
  20. 2026DynFly: Dynamic-Aware Continuous Trajectory Generation for UAV Vision-Language Navigation in Urban Environmentscited by 0
  21. 2026Vision Language Action (VLA) Models for Unmanned Aerial Robotics and Bimanual Manipulation: A Reviewcited by 0
  22. 2026ABot-AgentOS: A General Robotic Agent OS with Lifelong Multi-modal Memorycited by 0
  23. 2026Zero-Shot Mission-Level Evaluation for Aerial MLLM Agentscited by 0
  24. 2026RecoverFly: A Failure-Aware Reinforcement Learning Post-Training Framework for Aerial Vision-Language Navigationcited by 0
  25. 2026COSTA: A Cluster-Centric Paradigm for Annotation-Free Open-Set Semantic Segmentation of Aerial Point Clouds with Domain Shiftscited by 0
  26. 2026Dual-Layer Semantic-Spatial Belief Mapping for Aerial Object Goal Navigationcited by 0
  27. 2026OpenFlyScan: A Quality-Guided Aerial Reconstruction System for Consumer Dronescited by 0
  28. 2026Skytopia: Monocular Drone Navigation with Action-Conditioned Latent World Modelscited by 0
  29. 2026DroneWAM: Efficient World Action Model for Drone Visual Navigationcited by 0
  30. 2026Multi-Scale Semantic Mapping in Urban Environments via Observation Calibration and Policy Dependence Regularizationcited by 0
  31. 2026AerialDojo-200K: A Large-Scale Benchmark Suite for Open-World Aerial Object-Goal Searchcited by 0
  32. 2026LightVLN: Efficient Aerial Vision-and-Language Navigation with Compact Memory and History-Guided Local Aggregationcited by 0
  33. 2026Speaking the Navigator's Language: Trajectory-Grounded Instruction Translation for Frozen Aerial VLN Agentscited by 0
  34. 2026The Gap Is in the Geometry: A Convergent Meta-Analysis of Multi-Facet Aerial and Remote Sensing Video Understandingcited by 0