mlsanityData to 14 Oct 2026

Reinforcement learning with world model

arXiv 1908.11494 ↗First posted 30 Aug 2019cs.AIcited by 0

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Nowadays, model-free reinforcement learning algorithms have achieved remarkable performance on many decision making and control tasks, but high sample complexity and low sample efficiency still hinder the wide use of model-free reinforcement learning algorithms. In this paper, we argue that if we intend to design an intelligent agent that learns fast and transfers well, the agent must be able to reflect key elements of intelligence, like intuition, Memory, PredictionandCuriosity. We propose an agent framework that integrates off-policy reinforcement learning with world model learning, so as to embody the important features of intelligence in our algorithm design. We adopt the state-of-art model-free reinforcement learning algorithm, Soft Actor-Critic, as the agent intuition, and world model learning through RNN to endow the agent with memory, curiosity, and the ability to predict. We show that these ideas can work collaboratively with each other and our agent (RMC) can give new state-of-art results while maintaining sample efficiency and training stability. Moreover, our agent framework can be easily extended from MDP to POMDP problems without performance loss.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 10 of its 21 references that are in the map. First posted on arXiv 30 Aug 2019.

Builds on

10 of 21 references.

Closest ancestors · 3

Same direction, newest first

  1. 2018Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees10 Jul
  2. 2018World Models27 Mar
  3. 2018Model-Ensemble Trust-Region Policy Optimization28 Feb

Other references · 7

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Model-free Reinforcement Learning for Model-based Control: Towards Safe, Interpretable and Sample-efficient Agents
  2. 2024Reset-free Reinforcement Learning with World Models
  3. 2024Multi-agent Off-policy Actor-Critic Reinforcement Learning for Partially Observable Environments
  4. 2025General agents contain world models
  5. 2025Towards General-Purpose Model-Free Reinforcement Learning
  6. 2022A Unified Framework for Alternating Offline Model Training and Policy Learning
  7. 2026World Models as an Intermediary between Agents and the Real World
  8. 2026Puzzle it Out: Local-to-Global World Model for Offline Multi-Agent Reinforcement Learning