Towards Unified World Models for Visual Navigation via Memory-Augmented Planning and Foresight
On the map: World models → World models for navigation
Abstract (arXiv)
Enabling embodied agents to imagine future states is essential for robust and generalizable visual navigation. Yet, state-of-the-art systems typically rely on modular designs that decouple navigation planning from visual world modeling, which often induces state-action misalignment and weak adaptability in novel or dynamic scenarios. We propose UniWM, a unified, memory-augmented world model that integrates egocentric visual foresight and planning within a single multimodal autoregressive backbone. UniWM explicitly grounds action selection in visually imagined outcomes, tightly aligning prediction with control. Meanwhile, a hierarchical memory mechanism fuses short-term perceptual cues with longer-term trajectory context, supporting stable and coherent reasoning over extended horizons. Extensive experiments on four challenging benchmarks (Go Stanford, ReCon, SCAND, HuRoN) and the 1X Humanoid Dataset show that UniWM improves navigation success rates by up to 30%, substantially reduces trajectory errors against strong baselines, generalizes zero-shot to the unseen TartanDrive dataset, and scales naturally to high-dimensional humanoid navigation. These results position UniWM as a principled step toward unified, imagination-driven embodied navigation. The code and models are available at https://github.com/UWMILab/UniWM.
Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in World models for navigation, one of 11 directions in World models. Builds on 59 of its 80 references that are in the map. First posted on arXiv 9 Oct 2025.
Builds on
59 of 80 references.
Closest ancestors · 4
Same direction, newest first
- 2025NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environments30 Jun
- 2025Embodied AI Agents: Modeling the World27 Jun
- 2024Navigation World Models4 Dec
- 2021Pathdreamer: A World Model for Indoor Navigation18 May
Other references · 55
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2023Visual Instruction Tuningcited by 7,869
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
- 2018World Modelscited by 1,513
- 2023Mastering Diverse Domains through World Modelscited by 1,253
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
- 2020Mastering Atari with Discrete World Modelscited by 1,007
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
38 more
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2024Genie: Generative Interactive Environmentscited by 747
- 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
- 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
- 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
- 2025WorldVLA: Towards Autoregressive Action World Modelcited by 367
- 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 337
- 2020Learning to Explore using Active Neural SLAMcited by 316
- 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
- 2024Diffusion Models Are Real-Time Game Enginescited by 296
- 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Explorationcited by 248
- 2023ViNT: A Foundation Model for Visual Navigationcited by 222
- 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 206
- 2016Learning to Navigate in Complex Environmentscited by 198
- 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 191
- 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 185
- 2022GNM: A General Navigation Model to Drive Any Robotcited by 171
- 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 168
- 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
- 2025GameFactory: Creating New Games with Generative Interactive Videoscited by 130
- 2022Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigationcited by 127
- 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 121
- 2025Epona: Autoregressive Diffusion World Model for Autonomous Drivingcited by 117
- 2024ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generationcited by 114
- 2025Aether: Geometric-Aware Unified World Modelingcited by 97
- 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 79
- 20253D and 4D World Modeling: A Surveycited by 67
- 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPTcited by 66
- 2023SACSoN: Scalable Autonomous Control for Social Navigationcited by 61
- 2025Back to the Features: DINO as a Foundation for Video World Modelscited by 58
- 2021Coupling Vision and Proprioception for Navigation of Legged Robotscited by 51
- 2022TartanDrive: A Large-Scale Dataset for Learning Off-Road Dynamics Modelscited by 49
- 2023Fast Traversability Estimation for Wild Visual Navigationcited by 44
- 2024DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenescited by 38
- 2018GONet: A Semi-Supervised Deep Learning Approach For Traversability Estimationcited by 29
- 2025Critique of World Modelcited by 25
- 2024Human-Aware Vision-and-Language Navigation: Bridging Simulation to Reality with Dynamic Human Interactionscited by 19
- 2025LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequencescited by 17
50 most cited of 55 All 55 (JSON)
Built on by
Citing papers per year: 2025: 3; 2026: 11.
14 papers.
Sort:
By area:
- 2025Memory in the Age of AI Agentscited by 303
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 14
- 2025Aerial World Model for Long-horizon Visual Generation and Navigation in 3D Spacecited by 12
- 2026Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanationcited by 10
- 2025UrbanNav: Learning Language-Guided Urban Navigation from Web-Scale Human Trajectoriescited by 6
- 2026Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rolloutcited by 6
- 2026TwiFF (Think With Future Frames): A Large-Scale Dataset for Dynamic Visual Reasoningcited by 3
- 2026UniNav: A Unified World-Action Diffusion Model for Visual Navigationcited by 2
- 2026Meta-Modal Agent: Sequential Evidence Routing for Missing-Modality Candidate Rerankingcited by 0
- 2026Quo Vadis, World Modeling?cited by 0
- 2026Action Forcing: Training World Models on Unsupervised Video by Recovering Underlying Egomotion Basescited by 0
- 2026MemReg: Streaming Outdoor LiDAR Point Cloud Registration with Hybrid Memory Bufferscited by 0
2 more
- 2026VITAL: From Visual Motion to Precise Action14 Oct 2026
- 2026UniW(A)M: A Unified Mixture-of-Experts Framework for World and World-Action Models13 Oct 2026
- 2026MemReg: Streaming Outdoor LiDAR Point Cloud Registration with Hybrid Memory Buffers8 Oct 2026
- 2026Action Forcing: Training World Models on Unsupervised Video by Recovering Underlying Egomotion Bases24 Sep 2026
- 2026UniNav: A Unified World-Action Diffusion Model for Visual Navigation4 Aug 2026
- 2026Quo Vadis, World Modeling?3 Aug 2026
- 2026Meta-Modal Agent: Sequential Evidence Routing for Missing-Modality Candidate Reranking24 May 2026
- 2026Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rollout6 May 2026
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond24 Apr 2026
- 2026Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation20 Apr 2026
- 2026TwiFF (Think With Future Frames): A Large-Scale Dataset for Dynamic Visual Reasoning11 Feb 2026
- 2025Aerial World Model for Long-horizon Visual Generation and Navigation in 3D Space26 Dec 2025
2 more
- 2025Memory in the Age of AI Agents15 Dec 2025
- 2025UrbanNav: Learning Language-Guided Urban Navigation from Web-Scale Human Trajectories10 Dec 2025
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 14
- 2025Aerial World Model for Long-horizon Visual Generation and Navigation in 3D Spacecited by 12
- 2026Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanationcited by 10
- 2026Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rolloutcited by 6
- 2026UniNav: A Unified World-Action Diffusion Model for Visual Navigationcited by 2
- 2026Quo Vadis, World Modeling?cited by 0
- 2026Action Forcing: Training World Models on Unsupervised Video by Recovering Underlying Egomotion Basescited by 0
- 2026UniW(A)M: A Unified Mixture-of-Experts Framework for World and World-Action Modelscited by 0
- 2026VITAL: From Visual Motion to Precise Actioncited by 0
- 2026VITAL: From Visual Motion to Precise Action14 Oct 2026
- 2026UniW(A)M: A Unified Mixture-of-Experts Framework for World and World-Action Models13 Oct 2026
- 2026Action Forcing: Training World Models on Unsupervised Video by Recovering Underlying Egomotion Bases24 Sep 2026
- 2026UniNav: A Unified World-Action Diffusion Model for Visual Navigation4 Aug 2026
- 2026Quo Vadis, World Modeling?3 Aug 2026
- 2026Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rollout6 May 2026
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond24 Apr 2026
- 2026Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation20 Apr 2026
- 2025Aerial World Model for Long-horizon Visual Generation and Navigation in 3D Space26 Dec 2025
- 2025Memory in the Age of AI Agentscited by 303
- 2026Meta-Modal Agent: Sequential Evidence Routing for Missing-Modality Candidate Rerankingcited by 0
- 2026Meta-Modal Agent: Sequential Evidence Routing for Missing-Modality Candidate Reranking24 May 2026
- 2025Memory in the Age of AI Agents15 Dec 2025
Similar, not cited
Close in wording, no citation either way
- 2026NavWM: A Unified Navigation World Model for Foresight-Driven Planning
- 2026WAM-Nav: Asymmetric Latent World-Action Modeling for Unified Visual Navigation
- 2025AstraNav-World: World Model for Foresight Control and Consistency
- 2026NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation
- 2025NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Prediction
- 2026Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigation
- 2026LiteNWM: Efficient Latent World Models for Onboard Visual Navigation in the Wild
- 2025Expand Your SCOPE: Semantic Cognition over Potential-Based Exploration for Embodied Visual Navigation