mlsanityData to 14 Oct 2026

A Laplacian Framework for Option Discovery in Reinforcement Learning

arXiv 1703.00956 ↗First posted 2 Mar 2017cs.LG cs.AIcited by 110

Not on the map. Why?

Cited by 24 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Representation learning and option discovery are two of the biggest challenges in reinforcement learning (RL). Proto-value functions (PVFs) are a well-known approach for representation learning in MDPs. In this paper we address the option discovery problem by showing how PVFs implicitly define options. We do it by introducing eigenpurposes, intrinsic reward functions derived from the learned representations. The options discovered from eigenpurposes traverse the principal directions of the state space. They are useful for multiple tasks because they are discovered without taking the environment's rewards into consideration. Moreover, different options act at different time scales, making them helpful for exploration. We demonstrate features of eigenpurposes in traditional tabular domains as well as in Atari 2600 games.

Key facts. Cited by 110 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 14; 2022: 31; 2023: 21; 2024: 13; 2025: 11; 2026: 19.

110 papers.

Sort:

By area:

  1. 2023A Cookbook of Self-Supervised Learningcited by 250
  2. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 166
  3. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 141
  4. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 102
  5. 2024Foundation Policies with Hilbert Representationscited by 94
  6. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
  7. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 83
  8. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 73
  9. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 62
  10. 2021On The Effect of Auxiliary Tasks on Representation Dynamicscited by 57
  11. 2021Temporal Abstraction in Reinforcement Learning with the Successor Representationcited by 54
  12. 2021Hierarchical Skills for Efficient Explorationcited by 38
38 more
  1. 2022Spectral Decomposition Representation for Reinforcement Learningcited by 36
  2. 2018Learning to Play with Intrinsically-Motivated Self-Aware Agentscited by 35
  3. 2022A Survey on Recent Advances and Challenges in Reinforcement Learning Methods for Task-Oriented Dialogue Policy Learningcited by 30
  4. 2023Deep Laplacian-based Options for Temporally-Extended Explorationcited by 29
  5. 2022On the Generalization of Representations in Reinforcement Learningcited by 28
  6. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 28
  7. 2021Discovery of Options via Meta-Learned Subgoalscited by 27
  8. 2021Metrics and continuity in reinforcement learningcited by 26
  9. 2021Flexible Option Learningcited by 25
  10. 2024MaestroMotif: Skill Design from Artificial Intelligence Feedbackcited by 24
  11. 2021Towards Better Laplacian Representation in Reinforcement Learning with Generalized Graph Drawingcited by 23
  12. 2022Intelligent problem-solving as integrated hierarchical reinforcement learningcited by 21
  13. 2024Proto Successor Measure: Representing the Behavior Space of an RL Agentcited by 20
  14. 2021Direct then Diffuse: Incremental Unsupervised Skill Discovery for State Covering and Goal Reachingcited by 19
  15. 2022Reward-Respecting Subtasks for Model-Based Reinforcement Learningcited by 16
  16. 2022Unsupervised Learning of Temporal Abstractions with Slot-based Transformerscited by 14
  17. 2022Learning Dynamics and Generalization in Reinforcement Learningcited by 14
  18. 2022Generalised Policy Improvement with Geometric Policy Compositioncited by 13
  19. 2023Proper Laplacian Representation Learningcited by 11
  20. 2022MO2: Model-Based Offline Optionscited by 10
  21. 2022Reachability-Aware Laplacian Representation in Reinforcement Learningcited by 10
  22. 2025Mobi-π: Mobilizing Your Robot Learning Policycited by 10
  23. 2022Continual Auxiliary Task Learningcited by 9
  24. 2023Scalable Multi-agent Covering Option Discovery based on Kronecker Graphscited by 9
  25. 2021Average-Reward Learning and Planning with Optionscited by 8
  26. 2023Predictable MDP Abstraction for Unsupervised Model-Based RLcited by 8
  27. 2021DisTop: Discovering a Topological representation to learn diverse and rewarding skillscited by 7
  28. 2022Temporal Abstractions-Augmented Temporally Contrastive Learning: An Alternative to the Laplacian in RLcited by 7
  29. 2026Compositional Planning with Jumpy World Modelscited by 7
  30. 2023Granger Causal Interaction Skill Chainscited by 6
  31. 2023General Policies, Subgoal Structure, and Planning Widthcited by 6
  32. 2021Provable Hierarchy-Based Meta-Reinforcement Learningcited by 5
  33. 2022Unsupervised Reinforcement Learning for Transferable Manipulation Skill Discoverycited by 5
  34. 2023Creating Multi-Level Skill Hierarchies in Reinforcement Learningcited by 5
  35. 2026Laplacian Representations for Decision-Time Planningcited by 5
  36. 2022Learning Multi-agent Skills for Tabular Reinforcement Learning using Factor Graphscited by 4
  37. 2022Possibility Before Utility: Learning And Using Hierarchical Affordancescited by 4
  38. 2022Hierarchies of Reward Machinescited by 4

50 most cited of 110 All 110 (JSON)