# Self-supervised Hierarchical Visual Reasoning with World Model

arXiv: https://arxiv.org/abs/2605.17537 · HTML: https://mlsanity.com/paper/2605.17537/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in World models for language agents, one of 11 directions in World models.
- Builds on 20 of its 44 references that are in the map.
- First posted on arXiv 17 May 2026.

## Placed in

- World models / [World models for language agents](https://mlsanity.com/wm/world-models-for-language-agents/)
- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 20 of 20)

- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents](https://mlsanity.com/paper/2302.01560/) (2023)
- [Transformers are Sample-Efficient World Models](https://mlsanity.com/paper/2209.00588/) (2022)
- [World Simulation with Video Foundation Models for Physical AI](https://mlsanity.com/paper/2511.00062/) (2025)
- [JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models](https://mlsanity.com/paper/2311.05997/) (2023)
- [Transformer-based World Models Are Happy With 100k Interactions](https://mlsanity.com/paper/2303.07109/) (2023)
- [Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics](https://mlsanity.com/paper/2501.10100/) (2025)
- [Planning with Reasoning using Vision Language World Model](https://mlsanity.com/paper/2509.02722/) (2025)
- [3D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worlds](https://mlsanity.com/paper/2502.20041/) (2025)
- [JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse](https://mlsanity.com/paper/2503.16365/) (2025)
- [Fast-ThinkAct: Efficient Vision-Language-Action Reasoning via Verbalizable Latent Planning](https://mlsanity.com/paper/2601.09708/) (2026)
- [Fast ECoT: Efficient Embodied Chain-of-Thought via Thoughts Reuse](https://mlsanity.com/paper/2506.07639/) (2025)
- [Open-World Reinforcement Learning over Long Short-Term Imagination](https://mlsanity.com/paper/2410.03618/) (2024)
- [ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning and Online Reinforcement Learning](https://mlsanity.com/paper/2510.12693/) (2025)
- [ROCKET-2: Steering Visuomotor Policy via Cross-View Goal Alignment](https://mlsanity.com/paper/2503.02505/) (2025)
- [Open-World Skill Discovery from Unsegmented Demonstrations](https://mlsanity.com/paper/2503.10684/) (2025)
- [Bootstrap Off-policy with World Model](https://mlsanity.com/paper/2511.00423/) (2025)
- [Reflex First, Reflect Later: Latency-Aware Embodied LLM Agents for Dynamic Response](https://mlsanity.com/paper/2506.07223/) (2025)
- [Scalable Multi-Task Reinforcement Learning for Generalizable Spatial Intelligence in Visuomotor Agents](https://mlsanity.com/paper/2507.23698/) (2025)
- [COVR:Collaborative Optimization of VLMs and RL Agent for Visual-Based Control](https://mlsanity.com/paper/2601.06122/) (2026)

## Built on by (most cited first, 1 of 1)

- [A Unifying Perspective on Causal World Models: From Observations to Representations to Structure](https://mlsanity.com/paper/2608.13456/) (2026)

## Abstract (arXiv)

3D open-world environments with adversarial opponents remain a core challenge for reinforcement learning due to their vast state spaces. Effective reasoning representations are essential in such settings. While existing self-supervised visual foresight reasoning approaches often suffer from multi-step error accumulation, many recent studies resort to injecting domain-specific knowledge for more stable guidance. Our key insight is that the photorealistic fidelity of visual reasoning representations is secondary; what truly matters is providing informative, task-relevant signals. To this end, we propose ResDreamer, a hierarchical world model in which each higher-level layer is trained to reconstruct the residuals of the layer below. This design enables progressive abstraction of increasingly sophisticated world dynamics and fosters the emergence of richer latent representations. Drawing inspiration from the "Bitter Lesson", ResDreamer trains its reasoning representations in a purely self-supervised manner. The higher-level residual representations are used to modulate lower-level predictions, allowing the world model to scale effectively with only linearly increasing cross-layer communication costs. Experiments show that ResDreamer achieves state-of-the-art sample efficiency and parameter efficiency. This scalable hierarchical visual foresight reasoning architecture paves the way for more capable online RL agents in open-ended, dynamic environments. The code is accessible at https://github.com/XuYuanFei01/ResDreamer.