WorldMAP: Bootstrapping Vision-Language Navigation Trajectory Prediction with Generative World Models
On the map: World models → World models for navigation
also in 2 more
Abstract (arXiv)
Vision-language models (VLMs) and generative world models are opening new opportunities for embodied navigation. VLMs are increasingly used as direct planners or trajectory predictors, while world models support look-ahead reasoning by imagining future views. Yet predicting a reliable trajectory from a single egocentric observation remains challenging. Current VLMs often generate unstable trajectories, and world models, though able to synthesize plausible futures, do not directly provide the grounded signals needed for navigation learning. This raises a central question: how can generated futures be turned into supervision for grounded trajectory prediction? We present WorldMAP, a teacher--student framework that converts world-model-generated futures into persistent semantic-spatial structure and planning-derived supervision. Its world-model-driven teacher builds semantic-spatial memory from generated videos, grounds task-relevant targets and obstacles, and produces trajectory pseudo-labels through explicit planning. A lightweight student with a multi-hypothesis trajectory head is then trained to predict navigation trajectories directly from vision-language inputs. On Target-Bench, WorldMAP achieves the best ADE and FDE among compared methods, reducing ADE by 18.0% and FDE by 42.1% relative to the best competing baseline, while lifting a small open-source VLM to DTW performance competitive with proprietary models. More broadly, the results suggest that, in embodied navigation, the value of world models may lie less in supplying action-ready imagined evidence than in synthesizing structured supervision for navigation learning.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in World models for navigation, one of 11 directions in World models. Builds on 33 of its 39 references that are in the map. First posted on arXiv 9 Apr 2026.
Builds on
33 of 39 references.
Closest ancestors · 5
Same direction, newest first
- 2025NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environments30 Jun
- 2025WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigation4 Mar
- 2024Navigation World Models4 Dec
- 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigation14 Aug
- 2021Pathdreamer: A World Model for Indoor Navigation18 May
Other references · 28
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
- 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 983
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
- 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
- 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
- 2016Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Agecited by 405
- 2020Learning to Explore using Active Neural SLAMcited by 308
- 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
- 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 187
16 more
- 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179
- 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
- 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 125
- 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
- 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
- 2025Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigationcited by 56
- 2025MindJourney: Test-Time Scaling with World Models for Spatial Reasoningcited by 49
- 2023Improving Vision-and-Language Navigation by Generating Future-View Image Semanticscited by 46
- 2024ImagineNav: Prompting Vision-Language Models as Embodied Navigator through Scene Imaginationcited by 38
- 2024End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answeringcited by 37
- 2025UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoningcited by 27
- 2025Do Visual Imaginations Improve Vision-and-Language Navigation Agents?cited by 14
- 2025NaviTrace: Evaluating Embodied Navigation of Vision-Language Modelscited by 13
- 2026When and How Much to Imagine: Adaptive Test-Time Scaling with World Models for Visual Spatial Reasoningcited by 12
- 2025FSR-VLN: Fast and Slow Reasoning for Vision-Language Navigation with Hierarchical Multi-modal Scene Graphcited by 11
- 2025SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagerycited by 10
Built on by
Citing papers per year: 2026: 2.
2 papers.
- 2026UDAV: Uncertainty-Driven Adaptive VLM Waypoint Plannercited by 0
- 2026Rendering-Free Lookahead for Question-Guided Active Visioncited by 0
- 2026Rendering-Free Lookahead for Question-Guided Active Vision8 Oct 2026
- 2026UDAV: Uncertainty-Driven Adaptive VLM Waypoint Planner14 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2026FutureNav: Unified World-Action Modeling for Vision-and-Language Navigation
- 2025NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Prediction
- 2026World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning
- 2026NavWM: A Unified Navigation World Model for Foresight-Driven Planning
- 2025AstraNav-World: World Model for Foresight Control and Consistency
- 2026MWM: Mobile World Models for Action-Conditioned Consistent Prediction
- 2026UniNav: A Unified World-Action Diffusion Model for Visual Navigation
- 2026FINE: Future-Informed Navigation Encoding for Data-Efficient Vision-Language Navigation