mlsanityData to 7 Oct 2026

Simplifying Latent Dynamics with Softly State-Invariant World Models

arXiv 2401.17835 ↗First posted 31 Jan 2024cs.LGcited by 10

On the map: World models → Latent predictive world models

Abstract (arXiv)

To solve control problems via model-based reasoning or planning, an agent needs to know how its actions affect the state of the world. The actions an agent has at its disposal often change the state of the environment in systematic ways. However, existing techniques for world modelling do not guarantee that the effect of actions are represented in such systematic ways. We introduce the Parsimonious Latent Space Model (PLSM), a world model that regularizes the latent dynamics to make the effect of the agent's actions more predictable. Our approach minimizes the mutual information between latent states and the change that an action produces in the agent's latent state, in turn minimizing the dependence the state has on the dynamics. This makes the world model softly state-invariant. We combine PLSM with different model classes used for i) future latent state prediction, ii) planning, and iii) model-free reinforcement learning. We find that our regularization improves accuracy, generalization, and performance in downstream tasks, highlighting the importance of systematic treatment of actions in world models.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 34 of its 53 references that are in the map. First posted on arXiv 31 Jan 2024.

Builds on

34 of 53 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Temporal Difference Learning for Model Predictive Control9 Mar
  2. 2019Contrastive Learning of Structured World Models27 Nov
  3. 2018Learning Latent Dynamics for Planning from Pixels12 Nov

Other references · 31

  1. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  3. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  4. 2020Mastering Atari with Discrete World Modelscited by 990
  5. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  6. 2018DeepMind Control Suitecited by 935
  7. 2016Deep Variational Information Bottleneckcited by 915
  8. 2019RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Spacecited by 893
  9. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  10. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  11. 2019On Variational Bounds of Mutual Informationcited by 550
  12. 2023Learning Interactive Real-World Simulatorscited by 440
19 more
  1. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  2. 2022Transformers are Sample-Efficient World Modelscited by 317
  3. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  4. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  5. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 228
  6. 2017Emergence of Invariance and Disentanglement in Deep Representationscited by 213
  7. 2017DARLA: Improving Zero-Shot Transfer in Reinforcement Learningcited by 147
  8. 2019From Variational to Deterministic Autoencoderscited by 131
  9. 2018Generalization and Regularization in DQNcited by 117
  10. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 97
  11. 2019InfoBot: Transfer and Exploration via the Information Bottleneckcited by 78
  12. 2022Denoised MDPs: Learning World Models Better Than the World Itselfcited by 66
  13. 2022Understanding Self-Predictive Learning for Reinforcement Learningcited by 40
  14. 2021Robust Predictable Controlcited by 29
  15. 2021Dynamic Bottleneck for Robust Self-Supervised Explorationcited by 21
  16. 2023RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictabilitycited by 17
  17. 2023Reinforcement Learning with Simple Sequence Priorscited by 11
  18. 2023Efficient Dynamics Modeling in Interactive Environments with Koopman Theorycited by 10
  19. 2019Variational Predictive Information Bottleneckcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 8.

10 papers.

  1. 2026MRI Contrast Enhancement Kinetics World Modelcited by 4
  2. 2026Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challengescited by 3
  3. 2026ResWM: Residual-Action World Model for Visual RLcited by 2
  4. 2026World Model-Enabled Causal Digital Twins for Semantic Communications in Physical AI Systemscited by 2
  5. 2026DWM: Separating World Effects from Actions in Latent World Modelscited by 2
  6. 2025A circuit for predicting hierarchical structure in-context in Large Language Modelscited by 1
  7. 2026Learning Permutation-invariant Macroscopic Dynamicscited by 1
  8. 2025Prediction horizon shapes representations in predictive learningcited by 0
  9. 2026World Models in Pieces: Structural Certification for General Agentscited by 0
  10. 2026Dueling World Models: Advantage-Style Action Channels for Common-Mode Distractor Rejectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Latent State Design for World Models under Sufficiency Constraints
  2. 2026Modeling Latent Disturbances for Robust Decision-Making in World Models
  3. 2026Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning
  4. 2026ActSWM: Action-Sensitive World Models for Long-Horizon Planning in Open-World Games
  5. 2019Learning Dynamics Model in Reinforcement Learning by Incorporating the Long Term Future
  6. 2026LaWM: Least Action World Models for Long-Horizon Physical Consistency from Visual Observations
  7. 2026A Control Theory of Predictability in Latent World Models
  8. 2026Latent Geometry Beyond Search: Amortizing Planning in World Models