mlsanityData to 14 Oct 2026

Anticipate Before Acting: Future-State-Conditioned Vision-Language Navigation

arXiv 2607.18042 ↗First posted 20 Jul 2026cs.CV cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

End-to-end vision-language navigation (VLN) with causal vision-language models maps instructions and egocentric observations directly to actions, but standard behavior cloning supervises only the next action and does not explicitly encourage the policy state to be predictive of future visual outcomes, limiting long-horizon decision making. A privileged-input diagnostic shows that access to an expert-trajectory future image can substantially improve navigation, indicating that future observations contain rich, actionable cues, though such inputs are unavailable at deployment. Motivated by this signal, we propose Future-State-Conditioned VLN (FSC-VLN), a deployable model that augments a causal policy with a future-query token and uses training-only future-state supervision to distill information from future observations into the policy state. Concretely, during training we align the future-query representation to a frozen visual embedding Delta steps ahead, while inference requires only past and current observations. This design preserves the baseline inference pattern and adds only two learned prefix tokens, implying minimal overhead. On R2R val-unseen, FSC-VLN improves SR/OSR/SPL over a StreamVLN-style baseline under two training-data regimes, with larger gains on long-horizon episodes; ablations further support the dual-query design that separates future and action queries.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 18 references that are in the map. First posted on arXiv 20 Jul 2026.

Builds on

18 of 18 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026Causal World Modeling for Robot Control29 Jan

Other references · 17

  1. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  2. 2024Qwen2 Technical Reportcited by 2,120
  3. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  4. 2026World Action Models are Zero-shot Policiescited by 465
  5. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  6. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  7. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?cited by 349
  8. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 244
  9. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
  10. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 211
  11. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 156
  12. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 118
5 more
  1. 2026VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Modelcited by 118
  2. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
  3. 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 89
  4. 2025MapNav: A Novel Memory Representation via Annotated Semantic Maps for Vision-and-Language Navigationcited by 60
  5. 2026From Foundation to Application: Improving VLA Models in Practicecited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026FINE: Future-Informed Navigation Encoding for Data-Efficient Vision-Language Navigation
  2. 2026LatentPilot: Scene-Aware Vision-and-Language Navigation by Dreaming Ahead with Latent Visual Reasoning
  3. 2026FutureNav: Unified World-Action Modeling for Vision-and-Language Navigation
  4. 2025F1: A Vision-Language-Action Model Bridging Understanding and Generation to Actions
  5. 2026FoMoVLA: Bridging Visual Foresight and Motion Guidance for Vision-Language-Action Models
  6. 2026Policy-Guided World Model Planning for Language-Conditioned Visual Navigation
  7. 2026Language-Conditioned World Modeling for Visual Navigation
  8. 2026LookStep: Efficient Vision-Language Navigation with Linguistic Foresight and Event Driven Memory