mlsanityData to 14 Oct 2026

Learning World Models with Identifiable Factorization

arXiv 2306.06561 ↗First posted 11 Jun 2023cs.LGcited by 23

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Extracting a stable and compact representation of the environment is crucial for efficient reinforcement learning in high-dimensional, noisy, and non-stationary environments. Different categories of information coexist in such environments -- how to effectively extract and disentangle these information remains a challenging problem. In this paper, we propose IFactor, a general framework to model four distinct categories of latent state variables that capture various aspects of information within the RL system, based on their interactions with actions and rewards. Our analysis establishes block-wise identifiability of these latent variables, which not only provides a stable and compact representation but also discloses that all reward-relevant factors are significant for policy learning. We further present a practical approach to learning the world model with identifiable blocks, ensuring the removal of redundants but retaining minimal and sufficient information for policy optimization. Experiments in synthetic worlds demonstrate that our method accurately identifies the ground-truth latent variables, substantiating our theoretical findings. Moreover, experiments in variants of the DeepMind Control Suite and RoboDesk showcase the superior performance of our approach over baselines.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 41 of its 61 references that are in the map. First posted on arXiv 11 Jun 2023.

Builds on

41 of 61 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023Mastering Diverse Domains through World Models10 Jan
  2. 2022Denoised MDPs: Learning World Models Better Than the World Itself30 Jun
  3. 2022Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Models27 May
  4. 2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representations27 Oct
  5. 2020Mastering Atari with Discrete World Models5 Oct
  6. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  7. 2018World Models27 Mar
  8. 2015On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models30 Nov

Other references · 33

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  4. 2018DeepMind Control Suitecited by 956
  5. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  6. 2016Deep Variational Information Bottleneckcited by 930
  7. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  8. 2018MINE: Mutual Information Neural Estimationcited by 683
  9. 2018Disentangling by Factorisingcited by 616
  10. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  11. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  12. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 312
21 more
  1. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  2. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 273
  3. 2019Contrastive Learning of Structured World Modelscited by 209
  4. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 160
  5. 2019Scalable methods for computing state similarity in deterministic Markov Decision Processescited by 133
  6. 2022CITRIS: Causal Identifiability from Temporal Intervened Sequencescited by 133
  7. 2020Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Codingcited by 122
  8. 2021Learning Temporally Causal Latent Processes from General Temporal Datacited by 98
  9. 2022On the Identifiability of Nonlinear ICA: Sparsity and Beyondcited by 85
  10. 2020Deep Reinforcement and InfoMax Learningcited by 79
  11. 2021A Survey on Interpretable Reinforcement Learningcited by 73
  12. 2020Learning Predictive Representations for Deformable Objects Using Contrastive Estimationcited by 72
  13. 2022Temporally Disentangled Representation Learningcited by 61
  14. 2021Learning Task Informed Abstractionscited by 58
  15. 2020Predictive Information Accelerates Learning in RLcited by 56
  16. 2021Temporal Predictive Coding For Model-Based Planning In Latent Spacecited by 55
  17. 2022Causal Representation Learning for Instantaneous and Temporal Effects in Interactive Systemscited by 51
  18. 2015From Pixels to Torques: Policy Learning with Deep Dynamical Modelscited by 45
  19. 2021Action-Sufficient State Representation Learning for Control with Structural Constraintscited by 31
  20. 2022INFOrmation Prioritization through EmPOWERment in Visual Model-Based RLcited by 27
  21. 2022Learning Robust Dynamics through Variational Sparse Gatingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 7; 2026: 9.

23 papers.

Sort:

By area:

  1. 2025Latent Action Learning Requires Supervision in the Presence of Distractorscited by 44
  2. 2025NeuralOS: Towards Simulating Operating Systems via Neural Generative Modelscited by 19
  3. 2024Focus On What Matters: Separated Models For Visual-Based RL Generalizationcited by 6
  4. 2024DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractorscited by 5
  5. 2025Towards Empowerment Gain through Causal Structure Learning in Model-Based RLcited by 5
  6. 2026DDP-WM: Disentangled Dynamics Prediction for Efficient World Modelscited by 5
  7. 2025Modeling Unseen Environments with Language-guided Composable Causal Components in Reinforcement Learningcited by 4
  8. 2025Curious Causality-Seeking Agents Learn Meta Causal Worldcited by 4
  9. 2025From monoliths to modules: Decomposing transducers for efficient world modellingcited by 4
  10. 2024BECAUSE: Bilinear Causal Representation for Generalizable Offline Model-based Reinforcement Learningcited by 3
  11. 2025From Pixels to Factors: Learning Independently Controllable State Variables for Reinforcement Learningcited by 3
  12. 2024Towards Generalizable Reinforcement Learning via Causality-Guided Self-Adaptive Representationscited by 2
11 more
  1. 2026HaM-World: Soft-Hamiltonian World Models with Selective Memory for Planningcited by 2
  2. 2026Physically Viable World Models: A Case for Query-Conditioned Embodied AIcited by 2
  3. 2026Learning Task-Sufficient World Models by Synergizing Agentic Exploration and Structured Modelingcited by 2
  4. 2024SeMOPO: Learning High-quality Model and Policy from Low-quality Offline Visual Datasetscited by 1
  5. 2024Identifiable Representation and Model Learning for Latent Dynamic Systemscited by 1
  6. 2026Factored Causal Representation Learning for Robust Reward Modeling in RLHFcited by 1
  7. 2026DreamSAC: Learning Hamiltonian World Models via Symmetry Explorationcited by 1
  8. 2024Tracking control of latent dynamic systems with application to spacecraft attitude controlcited by 0
  9. 2026Dreaming Across Towns: Semantic Rollout and Town-Adversarial Regularization for Zero-Shot Held-Out-Town Fixed-Route Driving in CARLAcited by 0
  10. 2026From Generalist to Specialist Representationcited by 0
  11. 2026CausalDreamer: Learning Predictive World Models with Latent Disentanglementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Simplifying Latent Dynamics with Softly State-Invariant World Models
  2. 2026Inverting the Bellman Equation: From Q-Values to World Models
  3. 2026How do World Models and Policies Compose in LLM Agents? A Joint Spectral and Behavioral Account
  4. 2025Latent Action World Models for Control with Unlabeled Trajectories
  5. 2026Reward Prediction with Factorized World States
  6. 2026Beyond Prediction: Steering VLM Agents with Retrospective World Modeling
  7. 2026Latent State Design for World Models under Sufficiency Constraints
  8. 2026World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry