From Semantic Grounding to Decision Optimization: A Unified Framework for Long-Horizon UAV Vision-Language Navigation
Not on the map. Why?
Abstract (arXiv)
UAV vision-language navigation (UAV-VLN) focuses on enabling an aerial agent to follow natural-language instructions in open 3D environments from egocentric visual observations. Current approaches suffer from three coupled issues: weak grounding of instruction-relevant landmarks in visual observations, insufficient exploitation of long-horizon history, and unstable decisions under local traps or repeated exploration. To address these issues, we propose a unified semantic-to-decision framework. First, we present an instruction-grounded semantic enhancement module that injects object-level semantics and relative spatial cues into the current observation state. Subsequently, we develop a relevance-aware dynamic temporal aggregation strategy that reweights the full history buffer while converting a few high-relevance frames into structured landmark prompts for the decoder. Finally, we devise a topology-aware decision method that combines local-optimum cognition with group-relative policy optimization under progress, goal, semantic, and path-compliance rewards. Experiments on the widely used AerialVLN and OpenFly benchmarks clearly demonstrate that our method achieves state-of-the-art performance.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 32 of its 45 references that are in the map. First posted on arXiv 10 Aug 2026.
Builds on
32 of 45 references.
References · 32
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
- 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
- 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
- 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
- 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 121
- 2020Topological Planning with Transformers for Vision-and-Language Navigationcited by 116
- 2022MMRotate: A Rotated Object Detection Benchmark using PyTorchcited by 115
20 more
- 2020Language and Visual Entity Relationship Graph for Agent Navigationcited by 100
- 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 95
- 2025UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobilitycited by 49
- 2025NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environmentscited by 45
- 2024CityNav: A Large-Scale Dataset for Real-World Aerial Navigationcited by 42
- 2023Adaptive Sparse Convolutional Networks with Global Context Enhancement for Faster Object Detection on Drone Imagescited by 41
- 2021UFPMP-Det: Toward Accurate and Efficient Object Detection on Drone Imagerycited by 34
- 2025UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learningcited by 31
- 2025SmartWay: Enhanced Waypoint Prediction and Backtracking for Zero-Shot Vision-and-Language Navigationcited by 29
- 2025UAV-VLA: Vision-Language-Action System for Large Scale Aerial Mission Generationcited by 28
- 2025See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned Aerial Navigationcited by 18
- 2025GeoNav: Empowering MLLMs with dual-scale geospatial reasoning for language-goal aerial navigationcited by 16
- 2025NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Predictioncited by 16
- 2025SkyVLN: Vision-and-Language Navigation and NMPC Control for UAVs in Urban Environmentscited by 14
- 2025Aerial Vision-and-Language Navigation with Grid-based View Selection and Map Constructioncited by 12
- 2025UAV-VLN: End-to-End Vision Language guided Navigation for UAVscited by 11
- 2025LogisticsVLN: Vision-Language Navigation For Low-Altitude Terminal Delivery Based on Agentic UAVscited by 9
- 2025FlySearch: Exploring how vision-language models explorecited by 7
- 2026Fly0: Persistent Metric Anchoring for Zero-Shot Aerial Vision-Language Navigationcited by 4
- 2026DroneFINE: Domain-Aware Parameter-Efficient Fine-Tuning of Vision-Language Detectors for Drone Imagescited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.