mlsanityData to 7 Oct 2026

Visual Reinforcement Learning with Imagined Goals

arXiv 1807.04742 ↗First posted 12 Jul 2018cs.LG cs.CV cs.ROcited by 248

Not on the map. Why?

Cited by 25 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

For an autonomous agent to fulfill a wide range of user-specified goals at test time, it must be able to learn broadly applicable and general-purpose skill repertoires. Furthermore, to provide the requisite level of generality, these skills must handle raw sensory input such as images. In this paper, we propose an algorithm that acquires such general-purpose skills by combining unsupervised representation learning and reinforcement learning of goal-conditioned policies. Since the particular goals that might be required at test-time are not known in advance, the agent performs a self-supervised "practice" phase where it imagines goals and attempts to achieve them. We learn a visual representation with three distinct purposes: sampling goals for self-supervised practice, providing a structured transformation of raw sensory inputs, and computing a reward signal for goal reaching. We also propose a retroactive goal relabeling scheme to further improve the sample-efficiency of our method. Our off-policy algorithm is efficient enough to learn policies that operate on raw image observations and goals for a real-world robotic system, and substantially outperforms prior techniques.

Key facts. Cited by 248 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 4; 2021: 65; 2022: 56; 2023: 47; 2024: 32; 2025: 23; 2026: 20.

248 papers.

Sort:

By area:

  1. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  2. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  3. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  4. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 221
  5. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
  6. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 184
  7. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  8. 2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planningcited by 161
  9. 2019Goal-conditioned Imitation Learningcited by 159
  10. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  11. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
  12. 2021Discovering and Achieving Goals via World Modelscited by 131
38 more
  1. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  2. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 127
  3. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  4. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 117
  5. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
  6. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 107
  7. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
  8. 2022DALL-E-Bot: Introducing Web-Scale Diffusion Models to Roboticscited by 105
  9. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
  10. 2022VideoDex: Learning Dexterity from Internet Videoscited by 101
  11. 2022Deep Hierarchical Planning from Pixelscited by 96
  12. 2020Offline Reinforcement Learning from Images with Latent Space Modelscited by 94
  13. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 90
  14. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  15. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 71
  16. 2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 71
  17. 2022Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation?cited by 70
  18. 2022CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learningcited by 70
  19. 2023SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Executioncited by 69
  20. 2023Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agentscited by 66
  21. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learningcited by 64
  22. 2023Reinforcement Learning from Passive Data via Latent Intentionscited by 64
  23. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  24. 2021Offline Meta-Reinforcement Learning with Online Self-Supervisioncited by 52
  25. 2021Towards Robust Bisimulation Metric Learningcited by 49
  26. 2022Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulationcited by 49
  27. 2024Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformerscited by 47
  28. 2022Visual Reinforcement Learning with Self-Supervised 3D Representationscited by 45
  29. 2023An Unbiased Look at Datasets for Visuo-Motor Pre-Trainingcited by 41
  30. 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
  31. 2024HRP: Human Affordances for Robotic Pre-Trainingcited by 40
  32. 2023GenHowTo: Learning to Generate Actions and State Transformations from Instructional Videoscited by 39
  33. 2025DynaGuide: Steering Diffusion Polices with Active Dynamic Guidancecited by 39
  34. 2022Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learningcited by 38
  35. 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 36
  36. 2022Spectral Decomposition Representation for Reinforcement Learningcited by 35
  37. 2023Contrastive Difference Predictive Codingcited by 35
  38. 2024VLMPC: Vision-Language Model Predictive Control for Robotic Manipulationcited by 32

50 most cited of 248 All 248 (JSON)