# UDAV: Uncertainty-Driven Adaptive VLM Waypoint Planner

arXiv: https://arxiv.org/abs/2609.16368 · HTML: https://mlsanity.com/paper/2609.16368/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 18 references that are in the map.
- First posted on arXiv 14 Sep 2026.

## Builds on (most cited first, 12 of 12)

- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action](https://mlsanity.com/paper/2207.04429/) (2022)
- [PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs](https://mlsanity.com/paper/2402.07872/) (2024)
- [OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models](https://mlsanity.com/paper/2506.03135/) (2025)
- [GANav: Efficient Terrain Segmentation for Robot Navigation in Unstructured Outdoor Environments](https://mlsanity.com/paper/2103.04233/) (2021)
- [CoNVOI: Context-aware Navigation using Vision Language Models in Outdoor and Indoor Environments](https://mlsanity.com/paper/2403.15637/) (2024)
- [WayFAST: Navigation with Predictive Traversability in the Field](https://mlsanity.com/paper/2203.12071/) (2022)
- [Air-Ground Collaboration for Language-Specified Missions in Unknown Environments](https://mlsanity.com/paper/2505.09108/) (2025)
- [Enabling Large-scale Heterogeneous Collaboration with Opportunistic Communications](https://mlsanity.com/paper/2309.15975/) (2023)
- [CoFL: Continuous Flow Fields for Language-Conditioned Navigation](https://mlsanity.com/paper/2603.02854/) (2026)
- [WorldMAP: Bootstrapping Vision-Language Navigation Trajectory Prediction with Generative World Models](https://mlsanity.com/paper/2604.07957/) (2026)
- [AGT-CV: An Aerial-Ground Team Cross-View Dataset for Heterogeneous Robot Teams in Unstructured Environments](https://mlsanity.com/paper/2605.06478/) (2026)

## Abstract (arXiv)

Vision-language models (VLMs) can generate routes directly from aerial imagery for off-road navigation, but their predictions provide no indication of reliability. We present UDAV, an Uncertainty-Driven Adaptive VLM Waypoint Planner for UAV-guided UGV navigation. UDAV draws multiple stochastic trajectory predictions, selects their medoid as a self-consistent nominal route, and estimates predictive uncertainty from their spatial dispersion. When the maximum uncertainty across interior waypoints exceeds a threshold, UDAV invokes a reconsideration stage; otherwise, it returns the medoid directly. We evaluate UDAV on 400 held-out trajectory queries from two UAV flights. Stochastic medoid selection reduces the mean average displacement error (ADE) from 147.4 pixels for a deterministic prediction to 115.9 pixels. The complete planner achieves a mean ADE of 110.4 pixels, a 25.1% reduction relative to deterministic planning, while producing valid trajectories for all queries. UDAV also yields the lowest 90th- and 95th-percentile errors among all evaluated configurations, including a higher-budget K=10 consensus baseline. Relative to the K=5 medoid, UDAV reduces these errors from 225.3 and 326.0 pixels to 199.0 and 290.8 pixels, respectively. These results demonstrate that stochastic VLM predictions provide both a stronger nominal route and an actionable uncertainty signal for selectively mitigating large planning errors.