mlsanityData to 14 Oct 2026

See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned Aerial Navigation

arXiv 2509.22653 ↗First posted 26 Sep 2025cs.RO cs.AI cs.CLcited by 18

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

We present See, Point, Fly (SPF), a training-free aerial vision-and-language navigation (AVLN) framework built atop vision-language models (VLMs). SPF is capable of navigating to any goal based on any type of free-form instructions in any kind of environment. In contrast to existing VLM-based approaches that treat action prediction as a text generation task, our key insight is to consider action prediction for AVLN as a 2D spatial grounding task. SPF harnesses VLMs to decompose vague language instructions into iterative annotation of 2D waypoints on the input image. Along with the predicted traveling distance, SPF transforms predicted 2D waypoints into 3D displacement vectors as action commands for UAVs. Moreover, SPF also adaptively adjusts the traveling distance to facilitate more efficient navigation. Notably, SPF performs navigation in a closed-loop control manner, enabling UAVs to follow dynamic targets in dynamic environments. SPF sets a new state of the art in DRL simulation benchmark, outperforming the previous best method by an absolute margin of 63%. In extensive real-world evaluations, SPF outperforms strong baselines by a large margin. We also conduct comprehensive ablation studies to highlight the effectiveness of our design choice. Lastly, SPF shows remarkable generalization to different VLMs. Project page: https://spf-web.pages.dev

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 49 references that are in the map. First posted on arXiv 26 Sep 2025.

Builds on

34 of 49 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models3 Jun
  2. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs12 Feb
  3. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan

Other references · 31

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024Qwen2.5 Technical Reportcited by 4,942
  3. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  4. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
  5. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
  6. 2023Text2Motion: From Natural Language Instructions to Feasible Planscited by 224
  7. 2023RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketchescited by 133
  8. 2024MOKA: Open-World Robotic Manipulation through Mark-Based Visual Promptingcited by 129
  9. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 121
  10. 2023Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learningcited by 111
  11. 2018Mapping Instructions to Actions in 3D Environments with Visual Goal Predictioncited by 101
  12. 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 95
19 more
  1. 2017Control of a Quadrotor with Reinforcement Learningcited by 92
  2. 2018Neural Lander: Stable Drone Landing Control using Learned Dynamicscited by 85
  3. 2022Neural-Fly Enables Rapid Learning for Agile Flight in Strong Windscited by 85
  4. 2020Deep Drone Acrobaticscited by 75
  5. 2020AU-AIR: A Multi-modal Unmanned Aerial Vehicle Dataset for Low Altitude Traffic Surveillancecited by 56
  6. 2022Aerial Vision-and-Dialog Navigationcited by 49
  7. 2019Sim-to-(Multi)-Real: Transfer of Low-Level Robust Control Policies to Multiple Quadrotorscited by 45
  8. 2019Generalization through Simulation: Integrating Simulated and Real Data into Deep Reinforcement Learning for Vision-Based Autonomous Flightcited by 28
  9. 2025UAV-VLA: Vision-Language-Action System for Large Scale Aerial Mission Generationcited by 28
  10. 2018Following High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learningcited by 27
  11. 2023TypeFly: Flying Drones with Large Language Modelcited by 23
  12. 2025CognitiveDrone: A VLA Model and Evaluation Benchmark for Real-Time Cognitive Task Solving and Reasoning in UAVscited by 22
  13. 2017Toward Low-Flying Autonomous MAV Trail Navigation using Deep Neural Networks for Environmental Awarenesscited by 19
  14. 2024NavAgent: Multi-scale Urban Street View Fusion For UAV Embodied Vision-and-Language Navigationcited by 18
  15. 2023End-to-end Reinforcement Learning for Time-Optimal Quadcopter Flightcited by 17
  16. 2025GeoNav: Empowering MLLMs with dual-scale geospatial reasoning for language-goal aerial navigationcited by 16
  17. 2024DriveEnv-NeRF: Exploration of A NeRF-Based Autonomous Driving Environment for Real-World Performance Validationcited by 7
  18. 2024Flex: End-to-End Text-Instructed Visual Navigation from Foundation Model Featurescited by 3
  19. 2024Precise Pick-and-Place using Score-Based Diffusion Networkscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 14.

18 papers.

Sort:

By area:

  1. 2025VLA-AN: An Efficient and Onboard Vision-Language-Action Framework for Aerial Navigation in Complex Environmentscited by 23
  2. 2025LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integrationcited by 14
  3. 2026AerialVLA: A Vision-Language-Action Model for UAV Navigation via Minimalist End-to-End Controlcited by 11
  4. 2025Is your VLM Sky-Ready? A Comprehensive Spatial Intelligence Benchmark for UAV Navigationcited by 10
  5. 2026NavDreamer: Video Models as Zero-Shot 3D Navigatorscited by 9
  6. 2026AirHunt: Bridging VLM Semantics and Continuous Planning for Efficient Aerial Object Navigationcited by 4
  7. 2026Fly0: Persistent Metric Anchoring for Zero-Shot Aerial Vision-Language Navigationcited by 4
  8. 2026ESARBench: A Benchmark for Agentic UAV Embodied Search and Rescuecited by 4
  9. 2025Beyond Memorization: A Multi-Modal Ordinal Regression Benchmark to Expose Popularity Bias in Vision-Language Modelscited by 1
  10. 2026AeroDPO: Unleashing Lightweight UAV Navigation with High-Fidelity Perception and Automated Preference Optimizationcited by 1
  11. 2026From Semantic Grounding to Decision Optimization: A Unified Framework for Long-Horizon UAV Vision-Language Navigationcited by 1
  12. 2026Air-Ground Collaborative Vision-and-Language Navigation via Shared Bird's-Eye Mapscited by 1
6 more
  1. 2026Zero-Shot Mission-Level Evaluation for Aerial MLLM Agentscited by 0
  2. 2026RecoverFly: A Failure-Aware Reinforcement Learning Post-Training Framework for Aerial Vision-Language Navigationcited by 0
  3. 2026RACO: Reliability-Aware Coarse-Goal Optimization for Inspection-Oriented UAV Vision-Language Navigationcited by 0
  4. 2026Evaluating Multimodal LLMs as Generalist Vision-Language-Action Agents for Drone Control: Commanding, Approaching, Tracking and Searchingcited by 0
  5. 2026AirAnchor: Bridging Local and Global Spatial Information for Zero-Shot Aerial Vision-and-Language Navigationcited by 0
  6. 2026VLN on the Fly: An Onboard Vision-Language Navigation Stack for Aerial Robotscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Grounded Vision-Language Navigation for UAVs with Open-Vocabulary Goal Understanding
  2. 2025Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoning
  3. 2025OpenVLN: Open-world Aerial Vision-Language Navigation
  4. 2026AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wild
  5. 2026Vision-Language Navigation for Aerial Robots: Towards the Era of Large Language Models
  6. 2026Vision-and-Language Navigation for UAVs: Progress, Challenges, and a Research Roadmap
  7. 2025SoraNav: Adaptive UAV Task-Centric Navigation via Zeroshot VLM Reasoning
  8. 2026VLM-MPPI: Grounding Natural Language in Behaviorally Diverse Trajectories for Aerial Navigation ↗