# ActSWM: Action-Sensitive World Models for Long-Horizon Planning in Open-World Games

arXiv: https://arxiv.org/abs/2607.26712 · HTML: https://mlsanity.com/paper/2607.26712/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Latent predictive world models, one of 11 directions in World models.
- Builds on 23 of its 36 references that are in the map.
- First posted on arXiv 29 Jul 2026.

## Placed in

- World models / [Latent predictive world models](https://mlsanity.com/wm/latent-predictive-world-models/)

## Builds on (most cited first, 23 of 23)

- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion](https://mlsanity.com/paper/2407.01392/) (2024)
- [Revisiting Feature Prediction for Learning Visual Representations from Video](https://mlsanity.com/paper/2404.08471/) (2024)
- [TD-MPC2: Scalable, Robust World Models for Continuous Control](https://mlsanity.com/paper/2310.16828/) (2023)
- [MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge](https://mlsanity.com/paper/2206.08853/) (2022)
- [Temporal Difference Learning for Model Predictive Control](https://mlsanity.com/paper/2203.04955/) (2022)
- [DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning](https://mlsanity.com/paper/2411.04983/) (2024)
- [LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels](https://mlsanity.com/paper/2603.19312/) (2026)
- [Hierarchical Planning with Latent World Models](https://mlsanity.com/paper/2604.03208/) (2026)
- [Game-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game Agents](https://mlsanity.com/paper/2510.23691/) (2025)
- [Lumine: An Open Recipe for Building Generalist Agents in 3D Open Worlds](https://mlsanity.com/paper/2511.08892/) (2025)
- [Sensorimotor World Models: Perception for Action via Inverse Dynamics](https://mlsanity.com/paper/2606.20104/) (2026)
- [Delta-JEPA: Learning Action-Sensitive World Models via Latent Difference Decoding](https://mlsanity.com/paper/2606.31232/) (2026)
- [Open-World Reinforcement Learning over Long Short-Term Imagination](https://mlsanity.com/paper/2410.03618/) (2024)
- [MineStudio: A Streamlined Package for Minecraft AI Agent Development](https://mlsanity.com/paper/2412.18293/) (2024)
- [Co-Evolving Latent Action World Models](https://mlsanity.com/paper/2510.26433/) (2025)
- [World2Act: Latent Action Post-Training from World Model Dynamics](https://mlsanity.com/paper/2603.10422/) (2026)
- [Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning](https://mlsanity.com/paper/2606.21775/) (2026)
- [Enhancing Policy Learning with World-Action Model](https://mlsanity.com/paper/2603.28955/) (2026)
- [Latent Action World Models for Control with Unlabeled Trajectories](https://mlsanity.com/paper/2512.10016/) (2025)
- [Learning Actionable World Models for Industrial Process Control](https://mlsanity.com/paper/2503.01411/) (2025)
- [DWM: Separating World Effects from Actions in Latent World Models](https://mlsanity.com/paper/2607.18715/) (2026)

## Built on by (most cited first, 3 of 3)

- [Finite-Sample Metric Non-Collapse for Geometrically Supervised Latent World Models in Control](https://mlsanity.com/paper/2608.07265/) (2026)
- [Keeping JEPA World Models Plannable When Little of the Frame Moves](https://mlsanity.com/paper/2610.03137/) (2026)
- [When Low Prediction Error Misleads Planning: Diagnosing Representation, Dynamics, and Decision Failures in Latent World Models](https://mlsanity.com/paper/2610.05550/) (2026)

## Abstract (arXiv)

Latent world models support efficient model-predictive control by optimizing future control sequences in latent space and replanning in a receding-horizon manner. However, existing latent predictors often lack stable long-horizon rollout ability, and prediction accuracy alone does not ensure that rollouts remain responsive to the actions being planned. We identify Context Collapse, a failure mode in which autoregressive latent predictors maintain high similarity to future states while producing nearly indistinguishable futures under different action sequences. To address this issue, we propose ActSWM, an action-sensitive latent world model grounded in a transition-separation principle: a planning-useful latent dynamics model should keep alternative-action futures distinguishable and make the action associated with each local transition recoverable. Under this principle, action sensitivity is enforced as a constraint on latent rollouts rather than treated only as an auxiliary prediction target, encouraging predicted futures to preserve action-dependent differences over long horizons. Across step-drift analysis, closed-loop Minecraft planning, and cross-game local action recovery, ActSWM preserves larger action-dependent rollout gaps than existing baselines, improves task success in long-horizon interactive settings, and enables world-model-based action recovery from offline gameplay videos.