mlsanityData to 7 Oct 2026

Learning Latent Dynamic Robust Representations for World Models

arXiv 2405.06263 ↗First posted 10 May 2024cs.LG cs.AIcited by 20

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Visual Model-Based Reinforcement Learning (MBRL) promises to encapsulate agent's knowledge about the underlying dynamics of the environment, enabling learning a world model as a useful planner. However, top MBRL agents such as Dreamer often struggle with visual pixel-based inputs in the presence of exogenous or irrelevant noise in the observation space, due to failure to capture task-specific features while filtering out irrelevant spatio-temporal details. To tackle this problem, we apply a spatio-temporal masking strategy, a bisimulation principle, combined with latent reconstruction, to capture endogenous task-specific aspects of the environment for world models, effectively eliminating non-essential information. Joint training of representations, dynamics, and policy often leads to instabilities. To further address this issue, we develop a Hybrid Recurrent State-Space Model (HRSSM) structure, enhancing state representation robustness for effective policy learning. Our empirical evaluation demonstrates significant performance improvements over existing methods in a range of visually complex control tasks such as Maniskill citegu2023maniskill2 with exogenous distractors from the Matterport environment. Our code is avaliable at https://github.com/bit1029public/HRSSM.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 62 of its 75 references that are in the map. First posted on arXiv 10 May 2024.

Builds on

62 of 75 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023ReCoRe: Regularized Contrastive Representation Learning of World Model14 Dec
  2. 2023TD-MPC2: Scalable, Robust World Models for Continuous Control25 Oct
  3. 2023Mastering Diverse Domains through World Models10 Jan
  4. 2022Denoised MDPs: Learning World Models Better Than the World Itself30 Jun
  5. 2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representations27 Oct
  6. 2020Mastering Atari with Discrete World Models5 Oct
  7. 2020Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstruction29 Jul
  8. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  9. 2019When to Trust Your Model: Model-Based Policy Optimization19 Jun
  10. 2018Recurrent World Models Facilitate Policy Evolution4 Sep

Other references · 52

  1. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  4. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  5. 2017The Kinetics Human Action Video Datasetcited by 1,894
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  7. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  8. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  9. 2018Exploration by Random Network Distillationcited by 966
  10. 2018DeepMind Control Suitecited by 935
  11. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  12. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
38 more
  1. 2017Deep Reinforcement Learning that Matterscited by 727
  2. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  3. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  4. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  5. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  6. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  7. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  8. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  9. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  10. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
  11. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  12. 2020FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPscited by 232
  13. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 204
  14. 2022Masked World Models for Visual Controlcited by 169
  15. 2019Scalable methods for computing state similarity in deterministic Markov Decision Processescited by 133
  16. 2020Generalization in Reinforcement Learning by Soft Data Augmentationcited by 132
  17. 2020Invariant Causal Prediction for Block MDPscited by 104
  18. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
  19. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 97
  20. 2020Offline Reinforcement Learning from Images with Latent Space Modelscited by 94
  21. 2023Multi-View Masked World Models for Visual Robotic Manipulationcited by 78
  22. 2022Causal Dynamics Learning for Task-Independent State Abstractioncited by 64
  23. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrationscited by 59
  24. 2021Learning Task Informed Abstractionscited by 58
  25. 2019Combating the Compounding-Error Problem with a Multi-step Modelcited by 52
  26. 2021Towards Robust Bisimulation Metric Learningcited by 49
  27. 2022Masked Autoencoding for Scalable and Generalizable Decision Makingcited by 44
  28. 2020Generative Temporal Difference Learning for Infinite-Horizon Predictioncited by 43
  29. 2022Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Modelscited by 42
  30. 2019Learning to Combat Compounding-Error in Model-Based Reinforcement Learningcited by 39
  31. 2022Mask-based Latent Reconstruction for Reinforcement Learningcited by 39
  32. 2018Natural Environment Benchmarks for Reinforcement Learningcited by 32
  33. 2022Sample-Efficient Reinforcement Learning in the Presence of Exogenous Informationcited by 26
  34. 2023Simplified Temporal Consistency Reinforcement Learningcited by 17
  35. 2023RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictabilitycited by 17
  36. 2022Agent-Controller Representations: Principled Offline RL with Rich Exogenous Informationcited by 15
  37. 2021SimSR: Simple Distance-based State Representation for Deep Reinforcement Learningcited by 14
  38. 2022Behavior Prior Representation learning for Offline Reinforcement Learningcited by 12

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 14.

20 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey on World Models for Embodied AIcited by 65
  2. 2025FlowDreamer: A RGB-D World Model with Flow-based Motion Representations for Robot Manipulationcited by 17
  3. 2026Learning Invariant Visual Representations for Planning with Joint-Embedding Predictive World Modelscited by 11
  4. 2025DMWM: Dual-Mind World Model with Long-Term Imaginationcited by 10
  5. 2026EgoForge: Goal-Directed Egocentric World Simulatorcited by 9
  6. 2026WMAttack: Automated Attack Search for Adversarial Evaluation of World-Model Agentscited by 6
  7. 2025EvolvingAgent: Curriculum Self-evolving Agent with Continual World Model for Long-Horizon Taskscited by 5
  8. 2025World Model Robustness via Surprise Recognitioncited by 3
  9. 2026MetaMind: General and Cognitive World Models in Multi-Agent Systems by Meta-Theory of Mindcited by 3
  10. 2026PH-Dreamer: A Physics-Driven World Model via Port-Hamiltonian Generative Dynamicscited by 2
  11. 2026ARB4WM: An Adversarial Robustness Benchmark for World Models in Continuous Controlcited by 2
  12. 2026Diagnosing JEPA World Models with Action-Conditioned Predictive Consistencycited by 2
8 more
  1. 2026Debiased Model-based Representations for Sample-efficient Continuous Controlcited by 1
  2. 2026OrbiSim: World Models as Differentiable Physics Engines for Embodied Intelligencecited by 1
  3. 2025MInCo: Mitigating Information Conflicts in Distracted Visual Model-based Reinforcement Learningcited by 0
  4. 2026Pessimistic Auxiliary Policy for Offline Reinforcement Learningcited by 0
  5. 2026Towards Dual-Brain Minimal Sufficient Representation for Vision-Language Navigationcited by 0
  6. 2026TaskSense: Focusing on What Matters in World Modelscited by 0
  7. 2026Dueling World Models: Advantage-Style Action Channels for Common-Mode Distractor Rejectioncited by 0
  8. 2026StarWM: Self-Supervised Trained Attention Routing for Robust World Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Contrastive World Models
  2. 2025Zero-shot World Models via Search in Memory
  3. 2026Mask World Model: Predicting What Matters for Robust Robot Policy Learning
  4. 2021Temporal Predictive Coding For Model-Based Planning In Latent Space
  5. 2026Back to Parsimonious Latents: Learning Task-Centric World Models from Visual Foundations
  6. 2023Continual Visual Reinforcement Learning with A Life-Long World Model
  7. 2025Dynamics-Aligned Latent Imagination in Contextual World Models for Zero-Shot Generalization
  8. 2026FIRM-WM: State-factorized factual-interventional recurrent modeling for reward-free visual planning