mlsanityData to 13 Oct 2026

CURL: Contrastive Unsupervised Representations for Reinforcement Learning

arXiv 2004.04136 ↗First posted 8 Apr 2020cs.LG cs.CV stat.MLcited by 747

Not on the map. Why?

Cited by 150 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present CURL: Contrastive Unsupervised Representations for Reinforcement Learning. CURL extracts high-level features from raw pixels using contrastive learning and performs off-policy control on top of the extracted features. CURL outperforms prior pixel-based methods, both model-based and model-free, on complex tasks in the DeepMind Control Suite and Atari Games showing 1.9x and 1.2x performance gains at the 100K environment and interaction steps benchmarks respectively. On the DeepMind Control Suite, CURL is the first image-based algorithm to nearly match the sample-efficiency of methods that use state-based features. Our code is open-sourced and available at https://github.com/MishaLaskin/curl.

Key facts. Cited by 747 papers in the map (data through 13 Oct 2026). First posted on arXiv 8 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 158; 2022: 154; 2023: 163; 2024: 119; 2025: 79; 2026: 74.

747 papers.

Sort:

By area:

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,022
  2. 2023Mastering Diverse Domains through World Modelscited by 1,224
  3. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 790
  4. 2021Domain Generalization: A Surveycited by 760
  5. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 683
  6. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 491
  7. 2022Temporal Difference Learning for Model Predictive Controlcited by 384
  8. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 361
  9. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 357
  10. 2022Transformers are Sample-Efficient World Modelscited by 324
  11. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 293
  12. 2023A Cookbook of Self-Supervised Learningcited by 249
38 more
  1. 2022Masked Visual Pre-training for Motor Controlcited by 246
  2. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 244
  3. 2021Mastering Atari Games with Limited Datacited by 236
  4. 2021Reinforcement Learning with Prototypical Representationscited by 179
  5. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 176
  6. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  7. 2022Masked World Models for Visual Controlcited by 170
  8. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  9. 2023Language-Driven Representation Learning for Roboticscited by 163
  10. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 161
  11. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  12. 2023Getting aligned on representational alignmentcited by 142
  13. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  14. 2023Transformer-based World Models Are Happy With 100k Interactionscited by 138
  15. 2024Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goalscited by 132
  16. 2021Proof Artifact Co-training for Theorem Proving with Language Modelscited by 131
  17. 2021Representation Learning for Online and Offline RL in Low-rank MDPscited by 128
  18. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  19. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 114
  20. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  21. 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 113
  22. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  23. 2021APS: Active Pretraining with Successor Featurescited by 109
  24. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 107
  25. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 107
  26. 2021B-Pref: Benchmarking Preference-Based Reinforcement Learningcited by 106
  27. 2024ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulationcited by 106
  28. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
  29. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 103
  30. 2022Robust Mean Teacher for Continual and Gradual Test-Time Adaptationcited by 102
  31. 2023UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learningcited by 102
  32. 2023GNFactor: Multi-Task Real Robot Learning with Generalizable Neural Feature Fieldscited by 99
  33. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 98
  34. 2022SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learningcited by 96
  35. 2021Representation Matters: Offline Pretraining for Sequential Decision Makingcited by 90
  36. 2023STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learningcited by 88
  37. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentationcited by 86
  38. 2021S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learningcited by 85

50 most cited of 747 All 747 (JSON)