mlsanityData to 14 Oct 2026

Skew-Fit: State-Covering Self-Supervised Reinforcement Learning

arXiv 1903.03698 ↗First posted 8 Mar 2019cs.LG cs.AI cs.ROcited by 174

Not on the map. Why?

Cited by 39 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Autonomous agents that must exhibit flexible and broad capabilities will need to be equipped with large repertoires of skills. Defining each skill with a manually-designed reward function limits this repertoire and imposes a manual engineering burden. Self-supervised agents that set their own goals can automate this process, but designing appropriate goal setting objectives can be difficult, and often involves heuristic design decisions. In this paper, we propose a formal exploration objective for goal-reaching policies that maximizes state coverage. We show that this objective is equivalent to maximizing goal reaching performance together with the entropy of the goal distribution, where goals correspond to full state observations. To instantiate this principle, we present an algorithm called Skew-Fit for learning a maximum-entropy goal distributions. We prove that, under regularity conditions, Skew-Fit converges to a uniform distribution over the set of valid states, even when we do not know this set beforehand. Our experiments show that combining Skew-Fit for learning goal distributions with existing goal-reaching methods outperforms a variety of prior methods on open-sourced visual goal-reaching tasks. Moreover, we demonstrate that Skew-Fit enables a real-world robot to learn to open a door, entirely from scratch, from pixels, and without any manually-designed reward function.

Key facts. Cited by 174 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 37; 2022: 36; 2023: 35; 2024: 24; 2025: 26; 2026: 16.

174 papers.

Sort:

By area:

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  2. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  3. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 226
  4. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 182
  5. 2022Evolving Curricula with Regret-Based Environment Designcited by 173
  6. 2022Human-to-Robot Imitation in the Wildcited by 161
  7. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
  8. 2021Discovering and Achieving Goals via World Modelscited by 131
  9. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  10. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 121
  11. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 108
  12. 2022Deep Hierarchical Planning from Pixelscited by 101
38 more
  1. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
  2. 2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 73
  3. 2024Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Roboticscited by 73
  4. 2022CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learningcited by 71
  5. 2023Imitating Task and Motion Planning with Visuomotor Transformerscited by 66
  6. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 58
  7. 2022Winner Takes It All: Training Performant RL Populations for Combinatorial Optimizationcited by 58
  8. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 56
  9. 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
  10. 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
  11. 2022Spectral Decomposition Representation for Reinforcement Learningcited by 36
  12. 2021Autonomous Reinforcement Learning: Formalism and Benchmarkingcited by 33
  13. 2021Geometric Entropic Explorationcited by 32
  14. 2024Autonomous Improvement of Instruction Following Skills via Foundation Modelscited by 32
  15. 2022Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Spacecited by 31
  16. 2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 31
  17. 2024Adaptive Mobile Manipulation for Articulated Objects In the Open Worldcited by 31
  18. 2023Energy-based Models are Zero-Shot Planners for Compositional Scene Rearrangementcited by 30
  19. 2023Augmenting Autotelic Agents with Large Language Modelscited by 30
  20. 2024MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximizationcited by 30
  21. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 28
  22. 2023Planning Goals for Explorationcited by 28
  23. 2023Flipping Coins to Estimate Pseudocounts for Exploration in Reinforcement Learningcited by 28
  24. 2024Entity-Centric Reinforcement Learning for Object Manipulation from Pixelscited by 27
  25. 2024Grounding Video Models to Actions through Goal Conditioned Explorationcited by 26
  26. 2021Self-Supervised Euphemism Detection and Identification for Content Moderationcited by 25
  27. 2022Generalization with Lossy Affordances: Leveraging Broad Offline Data for Learning Visuomotor Taskscited by 25
  28. 2023Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shapingcited by 25
  29. 2021MURAL: Meta-Learning Uncertainty-Aware Rewards for Outcome-Driven Reinforcement Learningcited by 24
  30. 2023ALAN: Autonomously Exploring Robotic Agents in the Real Worldcited by 24
  31. 2021What Can I Do Here? Learning New Skills by Imagining Visual Affordancescited by 23
  32. 2021Successor Feature Landmarks for Long-Horizon Goal-Conditioned Reinforcement Learningcited by 23
  33. 2023Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Datacited by 23
  34. 2023Maximum State Entropy Exploration using Predecessor and Successor Representationscited by 23
  35. 2024A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoalscited by 23
  36. 2025Learning to Reason at the Frontier of Learnabilitycited by 23
  37. 2021Wasserstein Unsupervised Reinforcement Learningcited by 22
  38. 2022Action Noise in Off-Policy Deep Reinforcement Learning: Impact on Exploration and Performancecited by 22

50 most cited of 174 All 174 (JSON)