# Visual Cue Guided Video Planning for Generalizable Robot Navigation

arXiv: https://arxiv.org/abs/2609.16737 · HTML: https://mlsanity.com/paper/2609.16737/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 23 references that are in the map.
- First posted on arXiv 15 Sep 2026.

## Builds on (most cited first, 23 of 23)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion](https://mlsanity.com/paper/2407.01392/) (2024)
- [NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration](https://mlsanity.com/paper/2310.07896/) (2023)
- [StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modeling](https://mlsanity.com/paper/2507.05240/) (2025)
- [DeepMind Lab](https://mlsanity.com/paper/1612.03801/) (2016)
- [Embodied Navigation Foundation Model](https://mlsanity.com/paper/2509.12129/) (2025)
- [HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigation](https://mlsanity.com/paper/2409.14296/) (2024)
- [Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigation](https://mlsanity.com/paper/2512.08186/) (2025)
- [AllTracker: Efficient Dense Point Tracking at High Resolution](https://mlsanity.com/paper/2506.07310/) (2025)
- [TrackVLA: Embodied Visual Tracking in the Wild](https://mlsanity.com/paper/2505.23189/) (2025)
- [Enhancing Generalization in Vision-Language-Action Models by Preserving Pretrained Representations](https://mlsanity.com/paper/2509.11417/) (2025)
- [X-MOBILITY: End-To-End Generalizable Navigation via World Modeling](https://mlsanity.com/paper/2410.17491/) (2024)
- [Turning Video Models into Generalist Robot Policies](https://mlsanity.com/paper/2605.27817/) (2026)
- [Sparse Video Generation Propels Real-World Beyond-the-View Vision-Language Navigation](https://mlsanity.com/paper/2602.05827/) (2026)
- [NavDreamer: Video Models as Zero-Shot 3D Navigators](https://mlsanity.com/paper/2602.09765/) (2026)
- [ImagineUAV: Aerial Vision-Language Navigation via World-Action Modeling and Kinodynamic Planning](https://mlsanity.com/paper/2606.01205/) (2026)
- [ABot-N1: Toward a General Visual Language Navigation Foundation Model](https://mlsanity.com/paper/2607.10383/) (2026)
- [ImagiNav: Scalable Embodied Navigation via Generative Visual Prediction and Inverse Dynamics](https://mlsanity.com/paper/2603.13833/) (2026)
- [Robostral Navigate](https://mlsanity.com/paper/2607.20785/) (2026)
- [DreamToNav: Generalizable Navigation for Robots via Generative Video Planning](https://mlsanity.com/paper/2603.06190/) (2026)

## Abstract (arXiv)

Generative video models can serve as a promising backbone for robot navigation by predicting future observations as video plans. Recent approaches often condition video planning on short-horizon guidance and recover geometric waypoints through scene reconstruction, leaving longer-horizon planning and precise video-to-action translation less explored. We present CueNav, a video model-based navigation framework combining visual cue guided video planning with an embodiment-specific Inverse-Dynamics Model (IDM). As visual cues, we use a Bird's-Eye View (BEV) map to convey global task context and retain part of the robot body in the egocentric observation to expose embodiment context. These cues guide the video planner, while the IDM translates dense flow fields extracted from the video plan into robot actions. With the visual cue encoding global task context, CueNav achieves nearly 2x higher success in maze navigation than planning without the cue. The body-aware view with the IDM enables precise navigation with 70% success in a narrow passage where comparison methods largely fail to complete the task. We further demonstrate zero-shot semantic-conditioned navigation and deployment of the same video planner across different robot platforms. Our results show that visual cue-guided video planning with embodiment-specific action grounding paves the way toward a generalizable navigation framework for longer-horizon planning and embodiment-aware control. Additional results and code are available on our project website: https://cuenav.github.io.