mlsanityData to 14 Oct 2026

Going Beyond State-Reaching: Learning Abstractions for Intrinsically Motivated Option Discovery

arXiv 2609.36473 ↗First posted 29 Sep 2026cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Temporal abstraction via options can improve exploration in large environments. However, existing option discovery algorithms find subgoals that target all aspects of the state simultaneously. This state-reaching approach produces options that only apply in narrow regions of the state-space, eventually causing an explosion in the number of options that overwhelms the agent, and impedes progress on its primary task of reward maximization. We introduce an algorithm that instead identifies a small, relevant subset of features for each subgoal, yielding options that generalize broadly and accelerate exploration. Our approach learns abstract, transferrable options and achieves rapid exploration in three sparse-reward, image-based domains, including the Atari game MontezumasRevenge.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 42 of its 78 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

42 of 78 references.

References · 42

  1. 2023Segment Anythingcited by 8,234
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  3. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  4. 2017Hindsight Experience Replaycited by 1,136
  5. 2018Exploration by Random Network Distillationcited by 993
  6. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  7. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  8. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  9. 2016The Option-Critic Architecturecited by 448
  10. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  11. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  12. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 309
30 more
  1. 2020Integrated Task and Motion Planningcited by 290
  2. 2018Large-Scale Study of Curiosity-Driven Learningcited by 280
  3. 2016Variational Intrinsic Controlcited by 267
  4. 2020First return, then explorecited by 266
  5. 2018Semi-parametric Topological Memory for Navigationcited by 209
  6. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 174
  7. 2021Discovering and Achieving Goals via World Modelscited by 131
  8. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 122
  9. 2017A Laplacian Framework for Option Discovery in Reinforcement Learningcited by 110
  10. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  11. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 102
  12. 2022Deep Hierarchical Planning from Pixelscited by 101
  13. 2020Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Surveycited by 85
  14. 2018SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planningcited by 78
  15. 2020The Value-Improvement Path: Towards Better Representations for Reinforcement Learningcited by 60
  16. 2018PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Makingcited by 52
  17. 2022Predicate Invention for Bilevel Planningcited by 49
  18. 2021Learning Markov State Abstractions for Deep Reinforcement Learningcited by 44
  19. 2023Proto-Value Networks: Scaling Representation Learning with Auxiliary Taskscited by 42
  20. 2019Discovering Options for Exploration by Minimizing Cover Timecited by 31
  21. 2023Deep Laplacian-based Options for Temporally-Extended Explorationcited by 29
  22. 2022Human-level Atari 200x fastercited by 28
  23. 2023Flipping Coins to Estimate Pseudocounts for Exploration in Reinforcement Learningcited by 28
  24. 2018Many-Goals Reinforcement Learningcited by 22
  25. 2025Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesiscited by 22
  26. 2025Discovering Temporal Structure: An Overview of Hierarchical Reinforcement Learningcited by 22
  27. 2022The Alberta Plan for AI Researchcited by 18
  28. 2022Reward-Respecting Subtasks for Model-Based Reinforcement Learningcited by 16
  29. 2025From Pixels to Factors: Learning Independently Controllable State Variables for Reinforcement Learningcited by 3
  30. 2023Scaling Goal-based Exploration via Pruning Proto-goalscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Mastering Atari 2600 Games with Discovered Optionscited by 0