SpatialAnt: Autonomous Zero-Shot Robot Navigation via Active Scene Reconstruction and Visual Anticipation
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
Vision-and-Language Navigation (VLN) has recently benefited from Multimodal Large Language Models (MLLMs), enabling zero-shot navigation. While recent exploration-based zero-shot methods have shown promising results by leveraging global scene priors, they rely on high-quality human-crafted scene reconstructions, which are impractical for real-world robot deployment. When encountering an unseen environment, a robot should build its own priors through pre-exploration. However, these self-built reconstructions are inevitably incomplete and noisy, which severely degrade methods that depend on high-quality scene reconstructions. To address these issues, we propose SpatialAnt, a zero-shot navigation framework designed to bridge the gap between imperfect self-reconstructions and robust execution. SpatialAnt introduces a physical grounding strategy to recover the absolute metric scale for monocular-based reconstructions. Furthermore, rather than treating the noisy self-reconstructed scenes as absolute spatial references, we propose a novel visual anticipation mechanism. This mechanism leverages the noisy point clouds to render future observations, enabling the agent to perform counterfactual reasoning and prune paths that contradict human instructions. Extensive experiments in both simulated and real-world environments demonstrate that SpatialAnt significantly outperforms existing zero-shot methods. We achieve a 66% Success Rate (SR) on R2R-CE and 50.8% SR on RxR-CE benchmarks. Physical deployment on a Hello Robot further confirms the efficiency and efficacy of our framework, achieving a 52% SR in challenging real-world settings.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 37 references that are in the map. First posted on arXiv 27 Mar 2026.
Builds on
32 of 37 references.
Closest ancestors · 3
Same direction, newest first
- 2025Fast-SmartWay: Panoramic-Free End-to-End Zero-Shot Vision-and-Language Navigation2 Nov
- 2024ImagineNav: Prompting Vision-Language Models as Embodied Navigator through Scene Imagination13 Oct
- 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Models17 Jul
Other references · 29
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
- 2024Depth Anything V2cited by 1,385
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
- 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
- 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
- 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
- 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
- 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
- 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
17 more
- 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
- 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
- 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
- 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
- 2024SLAM3R: Real-Time Dense Scene Reconstruction from Monocular RGB Videoscited by 100
- 2021Structured Scene Memory for Vision-Language Navigationcited by 95
- 2024Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splattingcited by 73
- 2025Embodied Navigation Foundation Modelcited by 65
- 2024Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMscited by 60
- 2025MapNav: A Novel Memory Representation via Annotated Semantic Maps for Vision-and-Language Navigationcited by 55
- 2021The Design of Stretch: A Compact, Lightweight Mobile Manipulator for Indoor Human Environmentscited by 39
- 2025SmartWay: Enhanced Waypoint Prediction and Backtracking for Zero-Shot Vision-and-Language Navigationcited by 29
- 2021Curriculum Learning for Vision-and-Language Navigationcited by 18
- 2025VLN-Zero: Rapid Exploration and Cache-Enabled Neurosymbolic Vision-Language Planning for Zero-Shot Transfer in Robot Navigationcited by 8
- 2025STRIDER: Navigation via Instruction-Aligned Structural Decision Space Optimizationcited by 6
- 2026One Agent to Guide Them All: Empowering MLLMs for Vision-and-Language Navigation via Explicit World Representationcited by 5
- 2026Spatial-VLN: Zero-Shot Vision-and-Language Navigation With Explicit Spatial Perception and Explorationcited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026CATNAV: Cached Vision-Language Traversability for Efficient Zero-Shot Robot Navigation
- 2026OpenFrontier: General Navigation with Visual-Language Grounded Frontiers
- 2025RANGER: A Monocular Zero-Shot Semantic Navigation Framework through Visual Contextual Adaptation
- 2025Schr"odinger's Navigator: Imagining an Ensemble of Futures for Zero-Shot Object Navigation
- 2026Robostral Navigate
- 2026One MLLM, One Call: Efficient Zero-Shot Vision-and-Language Navigation via Spatial-Aware Waypoints
- 2025NavigateDiff: Visual Predictors are Zero-Shot Navigation Assistants
- 2023CorNav: Autonomous Agent with Self-Corrected Planning for Zero-Shot Vision-and-Language Navigation