mlsanityData to 7 Oct 2026

Latent State Design for World Models under Sufficiency Constraints

arXiv 2605.01694 ↗First posted 3 May 2026cs.AIcited by 3

On the map: World models → World models for language agents

Abstract (arXiv)

A world model matters to an agent only through the state it constructs. That state must preserve some information, discard other information, and support some future function: prediction, control, planning, memory, grounding, or counterfactual reasoning. This paper treats world-model research as latent state design under sufficiency constraints. We propose a functional taxonomy that groups methods by what their latent state is for, rather than by architecture or application domain: predictive embedding, recurrent belief state, object/causal structure, latent action interface, grounded planning interface, and memory substrate. These roles expose distinctions that architecture-based groupings hide, including the gap between predictive sufficiency and control sufficiency, and the gap between passive video prediction and counterfactual action modeling. The taxonomy supports an evaluation framework that judges a model by the sufficiency constraint its latent state was built to satisfy. We compare methods along seven axes: representation, prediction, planning, controllability, causal/counterfactual support, memory, and uncertainty. We use the resulting matrix as a diagnostic for what a latent state preserves, discards, and enables. The conclusion that follows is that an actionable world model is the one whose state construction matches the task, not the one that preserves the most information.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 50 of its 61 references that are in the map. First posted on arXiv 3 May 2026.

Builds on

50 of 61 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026CausalVAE as a Plug-in for World Models: Towards Reliable Counterfactual Dynamics9 Apr

Other references · 49

  1. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  7. 2023Mastering Diverse Domains through World Modelscited by 1,193
  8. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  9. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  10. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
  11. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  12. 2019Model-Based Reinforcement Learning for Ataricited by 495
37 more
  1. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  2. 2023Learning Interactive Real-World Simulatorscited by 440
  3. 2022Transformers are Sample-Efficient World Modelscited by 317
  4. 2024Latent Action Pretraining from Videoscited by 302
  5. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  6. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 237
  7. 2021Mastering Atari Games with Limited Datacited by 235
  8. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 178
  9. 2023Structured World Models from Human Videoscited by 144
  10. 2019Scalable methods for computing state similarity in deterministic Markov Decision Processescited by 133
  11. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 127
  12. 2025Video World Models with Long-term Spatial Memorycited by 124
  13. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
  14. 2026V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learningcited by 104
  15. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 89
  16. 2020The Value Equivalence Principle for Model-Based Reinforcement Learningcited by 88
  17. 2018Unsupervised Predictive Memory in a Goal-Directed Agentcited by 54
  18. 2021Temporal Predictive Coding For Model-Based Planning In Latent Spacecited by 54
  19. 2025Evaluating Newtonian Mechanics in Video Generative Models with Real Physical Systemscited by 44
  20. 2021MICo: Improved representations via sampling-based state similarity for Markov decision processescited by 43
  21. 2026Learning Latent Action World Models In The Wildcited by 42
  22. 2024Mastering Memory Tasks with World Modelscited by 38
  23. 2026Hierarchical Planning with Latent World Modelscited by 34
  24. 2026Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokenscited by 21
  25. 2025Can World Simulators Reason? Gen-ViRe: A Generative Visual Reasoning Benchmarkcited by 19
  26. 2025Value-guided action planning with JEPA world modelscited by 19
  27. 2026DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planningcited by 17
  28. 2025Object-Centric World Model for Language-Guided Manipulationcited by 16
  29. 2026World-Value-Action Model: Implicit Planning for Vision-Language-Action Systemscited by 13
  30. 2025Dyn-O: Building Structured World Models with Object-Centric Representationscited by 12
  31. 2025Learning Interactive World Model for Object-Centric Reinforcement Learningcited by 11
  32. 2026Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modelingcited by 11
  33. 2025Object-Centric World Models from Few-Shot Annotations for Sample-Efficient Reinforcement Learningcited by 9
  34. 2026Chain of World: World Model Thinking in Latent Motioncited by 8
  35. 2019Learning Latent State Spaces for Planning through Reward Predictioncited by 3
  36. 2026Object-Centric World Models Meet Monte Carlo Tree Searchcited by 1
  37. 2026Grounded World Model for Semantically Generalizable Planningcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Simplifying Latent Dynamics with Softly State-Invariant World Models
  2. 2026A Control Theory of Predictability in Latent World Models
  3. 2026DWM: Separating World Effects from Actions in Latent World Models
  4. 2026What Matters in Designing World Action Models: An Empirical Study
  5. 2026How do World Models and Policies Compose in LLM Agents? A Joint Spectral and Behavioral Account
  6. 2026MemWM: Memory-Augmented Text-Based World Model
  7. 2026From Generative Engines to Actionable Simulators: The Imperative of Physical Grounding in World Models
  8. 2026The Objective Is the Bottleneck: Latent World Models Encode What Their Planners Cannot Use