mlsanityData to 7 Oct 2026

Learning and Querying Fast Generative Models for Reinforcement Learning

arXiv 1802.03006 ↗First posted 8 Feb 2018cs.LGcited by 48

Not on the map. Why?

Abstract (arXiv)

A key challenge in model-based reinforcement learning (RL) is to synthesize computationally efficient and accurate environment models. We show that carefully designed generative models that learn and operate on compact state representations, so-called state-space models, substantially reduce the computational costs for predicting outcomes of sequences of actions. Extensive experiments establish that state-space models accurately capture the dynamics of Atari games from the Arcade Learning Environment from raw pixels. The computational speed-up of state-space models while maintaining high accuracy makes their application in RL feasible: We demonstrate that agents which query these models for decision making outperform strong model-free baselines on the game MSPACMAN, demonstrating the potential of using learned environment models for planning.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 15; 2022: 16; 2023: 4; 2024: 5; 2025: 3; 2026: 3.

48 papers.

Sort:

By area:

  1. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  2. 2021Discovering and Achieving Goals via World Modelscited by 131
  3. 2021Vector Quantized Models for Planningcited by 47
  4. 2021Clockwork Variational Autoencoderscited by 40
  5. 2022Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objectivecited by 28
  6. 2021Model-Based Reinforcement Learning via Latent-Space Collocationcited by 27
  7. 2022The Free Energy Principle for Perception and Action: A Deep Learning Perspectivecited by 24
  8. 2019Learning Belief Representations for Imitation Learning in POMDPscited by 23
  9. 2021Curiosity-Driven Exploration via Latent Bayesian Surprisecited by 22
  10. 2021Contrastive Active Inferencecited by 21
  11. 2021Neural Recursive Belief States in Multi-Agent Reinforcement Learningcited by 18
  12. 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Reviewcited by 18
36 more
  1. 2022Flow-based Recurrent Belief State Learning for POMDPscited by 16
  2. 2022Variational Causal Dynamics: Discovering Modular World Models from Interventionscited by 16
  3. 2022An Adaptive Deep RL Method for Non-Stationary Environments with Piecewise Stable Contextcited by 15
  4. 2020Towards Target-Driven Visual Navigation in Indoor Scenes via Generative Imitation Learningcited by 14
  5. 2021Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Modelscited by 14
  6. 2021High-Accuracy Model-Based Reinforcement Learning, a Surveycited by 13
  7. 2022Latent State Marginalization as a Low-cost Approach for Improving Explorationcited by 13
  8. 2022On Uncertainty in Deep State Space Models for Model-Based Reinforcement Learningcited by 13
  9. 2023Revisiting Structured Variational Autoencoderscited by 11
  10. 2023Informed POMDP: Leveraging Additional Information in Model-Based RLcited by 11
  11. 2024R-AIF: Solving Sparse-Reward Robotic Tasks from Pixels with Active Inference and World Modelscited by 10
  12. 2021Variational Predictive Routing with Nested Subjective Timescalescited by 9
  13. 2023On the Importance of Feature Decorrelation for Unsupervised Representation Learning in Reinforcement Learningcited by 8
  14. 2021Visual Perspective Taking for Opponent Behavior Modelingcited by 6
  15. 2022Mingling Foresight with Imagination: Model-Based Cooperative Multi-Agent Reinforcement Learningcited by 6
  16. 2022Deep Kernel Learning of Dynamical Models from High-Dimensional Noisy Datacited by 6
  17. 2021Learning State Representations via Retracing in Reinforcement Learningcited by 5
  18. 2022Safe Deep RL in 3D Environments using Human Feedbackcited by 5
  19. 2022Information-Theoretic Odometry Learningcited by 5
  20. 2025A Theoretical Justification for Asymmetric Actor-Critic Algorithmscited by 5
  21. 2021Learning What To Do by Simulating the Pastcited by 4
  22. 2022CtrlFormer: Learning Transferable State Representation for Visual Control via Transformercited by 4
  23. 2022LOPR: Latent Occupancy PRediction using Generative Modelscited by 4
  24. 2024Learning Abstract World Model for Value-preserving Planning with Optionscited by 4
  25. 2024An Empirical Study on the Power of Future Prediction in Partially Observable Environmentscited by 3
  26. 2025Informed Asymmetric Actor-Critic: Leveraging Privileged Signals Beyond Full-State Accesscited by 2
  27. 2022Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Modelscited by 1
  28. 2024Towards Cost Sensitive Decision Makingcited by 1
  29. 2021Predictive Control Using Learned State Space Models via Rolling Horizon Evolutioncited by 0
  30. 2021Dream to Explore: Adaptive Simulations for Autonomous Systemscited by 0
  31. 2022Online Planning in POMDPs with Self-Improving Simulatorscited by 0
  32. 2024Do Agents Dream of Electric Sheep?: Improving Generalization in Reinforcement Learning through Generative Learningcited by 0
  33. 2025Scaling Multi Agent Reinforcement Learning for Underwater Acoustic Tracking via Autonomous Vehiclescited by 0
  34. 2026Optimizing Neurorobot Policy under Limited Demonstration Data through Preference Regretcited by 0
  35. 2026Reinformed Dreamer: An Asymmetric World Model Efficiently Trained through Latent Guidancecited by 0
  36. 2026AtomWorld-Mem: Memory-Restored World States for Long-Horizon Atomistic Evolutioncited by 0