mlsanityData to 7 Oct 2026

PROSPECT: Unified Streaming Vision-Language Navigation via Semantic--Spatial Fusion and Latent Predictive Representation

arXiv 2603.03739 ↗First posted 4 Mar 2026cs.CV cs.AIcited by 2

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Multimodal large language models (MLLMs) have advanced zero-shot end-to-end Vision-Language Navigation (VLN), yet robust navigation requires not only semantic understanding but also predictive modeling of environment dynamics and spatial structure. We propose PROSPECT, a unified streaming navigation agent that couples a streaming Vision-Language-Action (VLA) policy with latent predictive representation learning. PROSPECT uses CUT3R as a streaming 3D foundation spatial encoder to produce long-context, absolute-scale spatial features, and fuses them with SigLIP semantic features via cross-attention. During training, we introduce learnable stream query tokens that query the streaming context and predict next-step 2D and 3D latent features (rather than pixels or explicit modalities), supervised in the latent spaces of frozen SigLIP and CUT3R teachers. The predictive branch shapes internal representations without inference overhead. Experiments on VLN-CE benchmarks and real-robot deployment demonstrate state-of-the-art performance and improved long-horizon robustness under diverse lighting. We will release code for the community soon.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 53 of its 58 references that are in the map. First posted on arXiv 4 Mar 2026.

Builds on

53 of 58 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence29 May
  2. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction26 May
  3. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec

Other references · 50

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  2. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  3. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  4. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  5. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  6. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  7. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  8. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  9. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  10. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  11. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  12. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
38 more
  1. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  2. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
  3. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  4. 2024Navigation World Modelscited by 262
  5. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  6. 2025Transfer between Modalities with MetaQueriescited by 231
  7. 2025Motus: A Unified Latent Action World Modelcited by 231
  8. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  9. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
  10. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  11. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  12. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
  13. 2025Streaming 4D Visual Geometry Transformercited by 147
  14. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 146
  15. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
  16. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 125
  17. 2025Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Modelcited by 125
  18. 2025Cambrian-S: Towards Spatial Supersensing in Videocited by 123
  19. 2025TTT3R: 3D Reconstruction as Test-Time Trainingcited by 122
  20. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
  21. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  22. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 107
  23. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  24. 2023GOAT: GO to Any Thingcited by 89
  25. 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 87
  26. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  27. 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 80
  28. 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 80
  29. 2025RynnVLA-002: A Unified Vision-Language-Action and World Modelcited by 70
  30. 2025HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generationcited by 56
  31. 2026InfiniteVGGT: Visual Geometry Grounded Transformer for Endless Streamscited by 51
  32. 2025NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environmentscited by 41
  33. 2024Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigationcited by 34
  34. 2025CorrectNav: Self-Correction Flywheel Empowers Vision-Language-Action Navigation Modelcited by 26
  35. 2025NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Predictioncited by 15
  36. 2025Mantis: A Versatile Vision-Language-Action Model with Disentangled Visual Foresightcited by 12
  37. 2025SparseWorld: A Flexible, Adaptive, and Efficient 4D Occupancy World Model Powered by Sparse and Dynamic Queriescited by 10
  38. 2024Aim My Robot: Precision Local Navigation to Any Objectcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026What Limits Vision-and-Language Navigation ?cited by 0
  2. 2026Physics-Guided Spatiotemporal State Space Modeling for Lookahead Molten Pool Segmentation in Laser Wire-Feed Weldingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026FINE: Future-Informed Navigation Encoding for Data-Efficient Vision-Language Navigation
  2. 2026SEDualVLN: A Spatially-Enhanced Dual-System for Vision-Language Navigation
  3. 2026GA-VLN: Geometry-Aware BEV Representation for Efficient Vision-Language Navigation
  4. 2025VLN-MME: Diagnosing MLLMs as Language-guided Visual Navigation agents
  5. 2026GemNav: Discrete-Token Visual Robot Navigation using a Multimodal Large Language Model
  6. 2025Efficient-VLN: A Simple yet Strong Baseline for Efficient Vision-Language Navigation
  7. 2025A Navigation Framework Utilizing Vision-Language Models
  8. 2026AgentVLN: Towards Agentic Vision-and-Language Navigation