mlsanityData to 14 Oct 2026

CURLing the Dream: Contrastive Representations for World Modeling in Reinforcement Learning

arXiv 2408.05781 ↗First posted 11 Aug 2024cs.LG cs.AI cs.CVcited by 0

On the map: World models → Latent dynamics models and model-based RL

also in 1 more
Abstract (arXiv)

In this work, we present Curled-Dreamer, a novel reinforcement learning algorithm that integrates contrastive learning into the DreamerV3 framework to enhance performance in visual reinforcement learning tasks. By incorporating the contrastive loss from the CURL algorithm and a reconstruction loss from autoencoder, Curled-Dreamer achieves significant improvements in various DeepMind Control Suite tasks. Our extensive experiments demonstrate that Curled-Dreamer consistently outperforms state-of-the-art algorithms, achieving higher mean and median scores across a diverse set of tasks. The results indicate that the proposed approach not only accelerates learning but also enhances the robustness of the learned policies. This work highlights the potential of combining different learning paradigms to achieve superior performance in reinforcement learning applications.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 21 of its 24 references that are in the map. First posted on arXiv 11 Aug 2024.

Builds on

21 of 24 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Mastering Diverse Domains through World Models10 Jan

Other references · 20

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  7. 2015Continuous control with deep reinforcement learningcited by 3,407
  8. 2018DeepMind Control Suitecited by 956
  9. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  10. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  11. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  12. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
8 more
  1. 2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learningcited by 35
  2. 2022Contrastive UCB: Provably Efficient Contrastive Self-Supervised Learning in Online Reinforcement Learningcited by 26
  3. 2023Behavior Contrastive Learning for Unsupervised Skill Discoverycited by 26
  4. 2020Contrastive Variational Reinforcement Learning for Complex Observationscited by 15
  5. 2022Depth-CUPRL: Depth-Imaged Contrastive Unsupervised Prioritized Representations in Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehiclescited by 8
  6. 2024Improving Generalization in Aerial and Terrestrial Mobile Robots Control Through Delayed Policy Learningcited by 2
  7. 2022Deep Reinforcement Learning Using a Low-Dimensional Observation Filter for Visual Complex Video Game Playingcited by 1
  8. 2023Model-enhanced Contrastive Reinforcement Learning for Sequential Recommendationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Multi-View Dreaming: Multi-View World Model with Contrastive Learning
  2. 2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representations
  3. 2019Dream to Control: Learning Behaviors by Latent Imagination
  4. 2022DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction
  5. 2020Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstruction
  6. 2020Bridging Imagination and Reality for Model-Based Deep Reinforcement Learning
  7. 2026Reinformed Dreamer: An Asymmetric World Model Efficiently Trained through Latent Guidance
  8. 2026CPPO: Contrastive Perception Policy Optimization for VLM Agents