# ZeST: A VLM-based Zero-Shot Traversability Navigation for Unknown Environments

arXiv: https://arxiv.org/abs/2508.19131 · HTML: https://mlsanity.com/paper/2508.19131/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 26 Aug 2025.

## Builds on (most cited first, 27 of 27)

- [LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action](https://mlsanity.com/paper/2207.04429/) (2022)
- [Visual Language Maps for Robot Navigation](https://mlsanity.com/paper/2210.05714/) (2022)
- [NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration](https://mlsanity.com/paper/2310.07896/) (2023)
- [BADGR: An Autonomous Self-Supervised Learning-Based Navigation System](https://mlsanity.com/paper/2002.05700/) (2020)
- [A2Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Models](https://mlsanity.com/paper/2308.07997/) (2023)
- [Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robots](https://mlsanity.com/paper/2405.01792/) (2024)
- [Fast Traversability Estimation for Wild Visual Navigation](https://mlsanity.com/paper/2305.08510/) (2023)
- [CoNVOI: Context-aware Navigation using Vision Language Models in Outdoor and Indoor Environments](https://mlsanity.com/paper/2403.15637/) (2024)
- [Bayesian Spatial Kernel Smoothing for Scalable Dense Semantic Mapping](https://mlsanity.com/paper/1909.04631/) (2019)
- [WayFAST: Navigation with Predictive Traversability in the Field](https://mlsanity.com/paper/2203.12071/) (2022)
- [Locomotion Policy Guided Traversability Learning using Volumetric Representations of Complex Environments](https://mlsanity.com/paper/2203.15854/) (2022)
- [STEP: Stochastic Traversability Evaluation and Planning for Risk-Aware Off-road Navigation](https://mlsanity.com/paper/2103.02828/) (2021)
- [EVORA: Deep Evidential Traversability Learning for Risk-Aware Off-Road Autonomy](https://mlsanity.com/paper/2311.06234/) (2023)
- [Risk-Aware Off-Road Navigation via a Learned Speed Distribution Map](https://mlsanity.com/paper/2203.13429/) (2022)
- [An Empirical Study Into What Matters for Calibrating Vision-Language Models](https://mlsanity.com/paper/2402.07417/) (2024)
- [Learned Visual Navigation for Under-Canopy Agricultural Robots](https://mlsanity.com/paper/2107.02792/) (2021)
- [Enabling Calibration In The Zero-Shot Inference of Large Vision-Language Models](https://mlsanity.com/paper/2303.12748/) (2023)
- [Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models](https://mlsanity.com/paper/2407.02666/) (2024)
- [Learning Risk-aware Costmaps for Traversability in Challenging Environments](https://mlsanity.com/paper/2107.11722/) (2021)
- [WayFASTER: a Self-Supervised Traversability Prediction for Increased Navigation Awareness](https://mlsanity.com/paper/2402.00683/) (2024)
- [A Survey of Traversability Estimation for Mobile Robots](https://mlsanity.com/paper/2204.10883/) (2022)
- [Evidential Semantic Mapping in Off-road Environments with Uncertainty-aware Bayesian Kernel Inference](https://mlsanity.com/paper/2403.14138/) (2024)
- [Risk-aware Path Planning via Probabilistic Fusion of Traversability Prediction for Planetary Rovers on Heterogeneous Terrains](https://mlsanity.com/paper/2303.01169/) (2023)
- [W-RIZZ: A Weakly-Supervised Framework for Relative Traversability Estimation in Mobile Robotics](https://mlsanity.com/paper/2406.02822/) (2024)
- [UNRealNet: Learning Uncertainty-Aware Navigation Features from High-Fidelity Scans of Real Environments](https://mlsanity.com/paper/2407.08720/) (2024)
- [Fed-EC: Bandwidth-Efficient Clustering-Based Federated Learning For Autonomous Visual Robot Navigation](https://mlsanity.com/paper/2411.04112/) (2024)
- [Lessons from Deploying CropFollow++: Under-Canopy Agricultural Navigation with Keypoints](https://mlsanity.com/paper/2404.17718/) (2024)

## Built on by (most cited first, 3 of 3)

- [TARIC: Memory-Augmented Traversability-Aware Outdoor VLN under Interrupted Semantic Cues](https://mlsanity.com/paper/2605.31121/) (2026)
- [Which Terrain Is Better? Preference Learning with VLM Prototypes for Off-Road Traversability Ranking](https://mlsanity.com/paper/2609.23673/) (2026)
- [From Language Priors to Field Adaptation: Preference Learning for Traversability Estimation](https://mlsanity.com/paper/2610.02974/) (2026)

## Abstract (arXiv)

Advances in robotics and autonomous navigation systems hinge on the ability to predict terrain traversability accurately. Traditional methods for generating datasets to train these prediction models often involve putting robots into potentially hazardous environments, posing risks to equipment and safety. To solve this problem, we present ZeST, a novel approach that treats repeated VLM outputs as stochastic measurements and fuses them into an uncertainty-aware posterior. Our approach not only performs zero-shot traversability and mitigates the risks associated with real-world data collection but also accelerates the development of advanced navigation systems, offering a cost-effective and scalable solution. To support our findings, we present navigation results, in both controlled indoor and unstructured outdoor environments. As shown in the experiments, ZeST provides safer navigation with a 90-100% success rate while no baseline exceeds 60%, and keeps complete success under query latencies up to 3.6s.