mlsanityData to 7 Oct 2026

ED2: Environment Dynamics Decomposition World Models for Continuous Control

arXiv 2112.02817 ↗First posted 6 Dec 2021cs.LG cs.AIcited by 0

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Model-based reinforcement learning (MBRL) achieves significant sample efficiency in practice in comparison to model-free RL, but its performance is often limited by the existence of model prediction error. To reduce the model error, standard MBRL approaches train a single well-designed network to fit the entire environment dynamics, but this wastes rich information on multiple sub-dynamics which can be modeled separately, allowing us to construct the world model more accurately. In this paper, we propose the Environment Dynamics Decomposition (ED2), a novel world model construction framework that models the environment in a decomposing manner. ED2 contains two key components: sub-dynamics discovery (SD2) and dynamics decomposition prediction (D2P). SD2 discovers the sub-dynamics in an environment automatically and then D2P constructs the decomposed world model following the sub-dynamics. ED2 can be easily combined with existing MBRL algorithms and empirical results show that ED2 significantly reduces the model error, increases the sample efficiency, and achieves higher asymptotic performance when combined with the state-of-the-art MBRL algorithms on various continuous control tasks. Our code is open source and available at https://github.com/ED2-source-code/ED2.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 39 of its 78 references that are in the map. First posted on arXiv 6 Dec 2021.

Builds on

39 of 78 references.

Closest ancestors · 10

Same direction, newest first

  1. 2020Bridging Imagination and Reality for Model-Based Deep Reinforcement Learning23 Oct
  2. 2020Mastering Atari with Discrete World Models5 Oct
  3. 2020Planning to Explore via Self-Supervised World Models12 May
  4. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  5. 2019When to Trust Your Model: Model-Based Policy Optimization19 Jun
  6. 2018Recurrent World Models Facilitate Policy Evolution4 Sep
  7. 2018Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees10 Jul
  8. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models30 May
  9. 2018Model-Ensemble Trust-Region Policy Optimization28 Feb
  10. 2017Value Prediction Network11 Jul

Other references · 29

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2016OpenAI Gymcited by 1,684
  6. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  8. 2018DeepMind Control Suitecited by 935
  9. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  10. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  11. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  12. 2019Model-Based Reinforcement Learning for Ataricited by 495
17 more
  1. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  2. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 220
  3. 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 167
  4. 2019Benchmarking Model-Based Reinforcement Learningcited by 158
  5. 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 152
  6. 2020Model-Based Offline Planningcited by 126
  7. 2018Model-Based Reinforcement Learning via Meta-Policy Optimizationcited by 99
  8. 2018Temporal Difference Models: Model-Free Deep RL for Model-Based Controlcited by 89
  9. 2019Combating the Compounding-Error Problem with a Multi-step Modelcited by 52
  10. 2020Goal-Aware Prediction: Learning to Model What Matterscited by 45
  11. 2019Variational Inference MPC for Bayesian Model-based Reinforcement Learningcited by 39
  12. 2020Bidirectional Model-based Policy Optimizationcited by 38
  13. 2020Trust the Model When It Is Confident: Masked Model-based Actor-Criticcited by 36
  14. 2020Measuring Visual Generalization in Continuous Control from Pixelscited by 20
  15. 2017Prediction and Control with Temporal Segment Modelscited by 14
  16. 2019Model Imitation for Model-Based Reinforcement Learningcited by 4
  17. 2017Crossmodal Attentive Skill Learnercited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Models
  2. 2020Ready Policy One: World Building Through Active Learning
  3. 2026EnvRL: Learn from Environment Dynamics in Agentic Reinforcement Learning
  4. 2022A Survey on Model-based Reinforcement Learning
  5. 2024PWM: Policy Learning with Multi-Task World Models
  6. 2025Policy-Driven World Model Adaptation for Robust Offline Model-based Reinforcement Learning
  7. 2026DWM: Separating World Effects from Actions in Latent World Models
  8. 2024Exploring the limits of Hierarchical World Models in Reinforcement Learning