# LookStep: Efficient Vision-Language Navigation with Linguistic Foresight and Event Driven Memory

arXiv: https://arxiv.org/abs/2609.02350 · HTML: https://mlsanity.com/paper/2609.02350/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 73 references that are in the map.
- First posted on arXiv 2 Sep 2026.

## Builds on (most cited first, 25 of 25)

- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2004.02857/) (2020)
- [NaVILA: Legged Robot Vision-Language-Action Model for Navigation](https://mlsanity.com/paper/2412.04453/) (2024)
- [Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoning](https://mlsanity.com/paper/2403.11401/) (2024)
- [Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Tasks](https://mlsanity.com/paper/2412.06224/) (2024)
- [StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modeling](https://mlsanity.com/paper/2507.05240/) (2025)
- [JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigation](https://mlsanity.com/paper/2509.22548/) (2025)
- [Introduction to Latent Variable Energy-Based Models: A Path Towards Autonomous Machine Intelligence](https://mlsanity.com/paper/2306.02572/) (2023)
- [NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Prediction](https://mlsanity.com/paper/2512.01550/) (2025)
- [MaPPO: Maximum a Posteriori Preference Optimization with Prior Knowledge](https://mlsanity.com/paper/2507.21183/) (2025)
- [PriorDrive: Enhancing Online HD Mapping with Unified Vector Priors](https://mlsanity.com/paper/2409.05352/) (2024)
- [CE-Nav: Flow-Guided Reinforcement Refinement for Cross-Embodiment Local Navigation](https://mlsanity.com/paper/2509.23203/) (2025)
- [Nav-R2 Dual-Relation Reasoning for Generalizable Open-Vocabulary Object-Goal Navigation](https://mlsanity.com/paper/2512.02400/) (2025)
- [Lifting Traces to Logic: Programmatic Skill Induction with Neuro-Symbolic Learning for Long-Horizon Agentic Tasks](https://mlsanity.com/paper/2605.01293/) (2026)
- [Scaling Multi-Camera 3D Object Detection through Weak-to-Strong Eliciting](https://mlsanity.com/paper/2404.06700/) (2024)
- [Stability Under Scrutiny: Benchmarking Representation Paradigms for Online HD Mapping](https://mlsanity.com/paper/2510.10660/) (2025)
- [MLP-SLAM: Multilayer Perceptron-Based Simultaneous Localization and Mapping](https://mlsanity.com/paper/2410.10669/) (2024)
- [Self-Evolving Neuro-Symbolic Skills for Tool-Augmented Spatial Reasoning](https://mlsanity.com/paper/2608.07955/) (2026)
- [Thinking with Tables: Enhancing Multi-Modal Tabular Understanding via Neuro-Symbolic Reasoning](https://mlsanity.com/paper/2603.24004/) (2026)
- [LAST: Leveraging Tools as Hints to Enhance Spatial Reasoning for Multimodal Large Language Models](https://mlsanity.com/paper/2604.09712/) (2026)
- [Learning for Long-Horizon Planning via Neuro-Symbolic Abductive Imitation](https://mlsanity.com/paper/2411.18201/) (2024)
- [MerNav: A Highly Generalizable Memory-Execute-Review Framework for Zero-Shot Object Goal Navigation](https://mlsanity.com/paper/2602.05467/) (2026)
- [NeSy-Route: A Neuro-Symbolic Benchmark for Constrained Route Planning in Remote Sensing](https://mlsanity.com/paper/2603.16307/) (2026)
- [Memory-Augmented Vision-Language Agents for Persistent and Semantically Consistent Object Captioning](https://mlsanity.com/paper/2603.24257/) (2026)

## Built on by (most cited first, 1 of 1)

- [HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness](https://mlsanity.com/paper/2609.15195/) (2026)

## Abstract (arXiv)

Vision-Language Navigation (VLN) requires an embodied agent to follow natural-language instructions in unseen environments. Recent progress has been largely driven by Multimodal Large Language Models (MLLMs). Existing methods follow a next-step action prediction paradigm, supervising only the expert action, which requires a high quantity of data for training. They also rely on cognitive maps, accumulated historical frames, or external 3D tools to maintain states, leading to high computational and memory overhead. To realize resource efficiency VLN, we propose LookStep, a unified end-to-end framework that combines Language Centric Future State Modeling and Event Driven Rolling Memory that uses language labels to generate coarse-grained navigation progress and future states for each candidate action, while autonomously deciding whether to write each observation into a bounded rolling memory with a semantic role. We validate LookStep empirically. On VLN-CE tasks, LookStep outperforms existing methods under the same training settings, achieving a 49.7% success rate on R2R-CE Val-Unseen with better memory efficiency and less data usage. Code and model is available at https://github.com/kunyang-YU/LookStep.