# NavThinker: Action-Conditioned World Models for Coupled Prediction and Planning in Social Navigation

arXiv: https://arxiv.org/abs/2603.15359 · HTML: https://mlsanity.com/paper/2603.15359/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in World models for navigation, one of 11 directions in World models.
- Builds on 11 of its 38 references that are in the map.
- First posted on arXiv 16 Mar 2026.

## Placed in

- World models / [World models for navigation](https://mlsanity.com/wm/world-models-for-navigation/)

## Builds on (most cited first, 11 of 11)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [When to Trust Your Model: Model-Based Policy Optimization](https://mlsanity.com/paper/1906.08253/) (2019)
- [DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning](https://mlsanity.com/paper/2411.04983/) (2024)
- [Diffusion for World Modeling: Visual Details Matter in Atari](https://mlsanity.com/paper/2405.12399/) (2024)
- [STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learning](https://mlsanity.com/paper/2310.09615/) (2023)
- [X-MOBILITY: End-To-End Generalizable Navigation via World Modeling](https://mlsanity.com/paper/2410.17491/) (2024)

## Built on by (most cited first, 6 of 6)

- [GeoSem-WAM: Geometry- and Semantic-Aware World Action Models](https://mlsanity.com/paper/2606.03188/) (2026)
- [Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation](https://mlsanity.com/paper/2604.18486/) (2026)
- [Think When It Matters: Conditional VLM Reasoning for Social Navigation with RL Policies](https://mlsanity.com/paper/2607.10991/) (2026)
- [Quo Vadis, World Modeling?](https://mlsanity.com/paper/2608.02713/) (2026)
- [Feeling Terrain Before Crossing: World Models for Off-Road Navigation](https://mlsanity.com/paper/2609.19863/) (2026)
- [Social-WM: Safety-Aware Latent World Models for Robot Social Navigation](https://mlsanity.com/paper/2609.40177/) (2026)

## Abstract (arXiv)

Social navigation requires robots to act safely in dynamic human environments. Effective behavior demands thinking ahead: reasoning about how the scene and pedestrians evolve under different robot actions rather than reacting to current observations alone. This creates a coupled prediction-planning challenge, where robot actions and human motion mutually influence each other. To address this challenge, we propose NavThinker, a future-aware framework that couples an action-conditioned world model with on-policy reinforcement learning. The world model operates in the Depth Anything V2 patch feature space and performs autoregressive prediction of future scene geometry and human motion; multi-head decoders then produce future depth maps and human trajectories, yielding a future-aware state aligned with traversability and interaction risk. Crucially, we train the policy with DD-PPO while injecting world-model think-ahead signals via: (i) action-conditioned future features fused into the current observation embedding and (ii) social reward shaping from predicted human trajectories. Experiments on single- and multi-robot Social-HM3D show state-of-the-art navigation success, with zero-shot transfer to Social-MP3D and real-world deployment on a Unitree Go2, validating generalization and practical applicability. Webpage: https://hutslib.github.io/NavThinker.