# SmallWorlds: Assessing Dynamics Understanding of World Models in Isolated Environments

arXiv: https://arxiv.org/abs/2511.23465 · HTML: https://mlsanity.com/paper/2511.23465/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Latent predictive world models, one of 11 directions in World models.
- Builds on 25 of its 35 references that are in the map.
- First posted on arXiv 28 Nov 2025.

## Placed in

- World models / [Latent predictive world models](https://mlsanity.com/wm/latent-predictive-world-models/)

## Builds on (most cited first, 25 of 25)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [Genie: Generative Interactive Environments](https://mlsanity.com/paper/2402.15391/) (2024)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion](https://mlsanity.com/paper/2407.01392/) (2024)
- [dm_control: Software and Tasks for Continuous Control](https://mlsanity.com/paper/2006.12983/) (2020)
- [Transformers are Sample-Efficient World Models](https://mlsanity.com/paper/2209.00588/) (2022)
- [Diffusion for World Modeling: Visual Details Matter in Atari](https://mlsanity.com/paper/2405.12399/) (2024)
- [MineRL: A Large-Scale Dataset of Minecraft Demonstrations](https://mlsanity.com/paper/1907.13440/) (2019)
- [How Far is Video Generation from World Model: A Physical Law Perspective](https://mlsanity.com/paper/2411.02385/) (2024)
- [Matrix-game 2.0: An open-source real-time and streaming interactive world model](https://mlsanity.com/paper/2508.13009/) (2025)
- [Masked World Models for Visual Control](https://mlsanity.com/paper/2206.14244/) (2022)
- [Structured World Models from Human Videos](https://mlsanity.com/paper/2308.10901/) (2023)
- [PHYRE: A New Benchmark for Physical Reasoning](https://mlsanity.com/paper/1908.05656/) (2019)
- [STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learning](https://mlsanity.com/paper/2310.09615/) (2023)
- [Reasoning About Physical Interactions with Object-Oriented Prediction and Planning](https://mlsanity.com/paper/1812.10972/) (2018)
- [Learning Long-term Visual Dynamics with Region Proposal Interaction Networks](https://mlsanity.com/paper/2008.02265/) (2020)
- [Evaluating Long-Term Memory in 3D Mazes](https://mlsanity.com/paper/2210.13383/) (2022)
- [Neural Motion Simulator: Pushing the Limit of World Models in Reinforcement Learning](https://mlsanity.com/paper/2504.07095/) (2025)
- [Towards Unraveling and Improving Generalization in World Models](https://mlsanity.com/paper/2501.00195/) (2024)

## Built on by (most cited first, 4 of 4)

- [Out of Sight, Out of Mind? Evaluating State Evolution in Video World Models](https://mlsanity.com/paper/2603.13215/) (2026)
- [Understanding Rollout Error in Graph World Models](https://mlsanity.com/paper/2606.27780/) (2026)
- [BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs](https://mlsanity.com/paper/2605.30900/) (2026)
- [Beyond Myopic World Models: Long-Horizon End-to-End Training for Direct Future Prediction](https://mlsanity.com/paper/2608.07420/) (2026)

## Abstract (arXiv)

Current world models lack a unified and controlled setting for systematic evaluation, making it difficult to assess whether they truly capture the underlying rules that govern environment dynamics. In this work, we address this open challenge by introducing the SmallWorld Benchmark, a testbed designed to assess world model capability under isolated and precisely controlled dynamics without relying on handcrafted reward signals. Using this benchmark, we conduct comprehensive experiments in the fully observable state space on representative architectures including Recurrent State Space Model, Transformer, Diffusion model, and Neural ODE, examining their behavior across six distinct domains. The experimental results reveal how effectively these models capture environment structure and how their predictions deteriorate over extended rollouts, highlighting both the strengths and limitations of current modeling paradigms and offering insights into future improvement directions in representation learning and dynamics modeling.