mlsanityData to 7 Oct 2026

Proper Laplacian Representation Learning

arXiv 2310.10833 ↗First posted 16 Oct 2023cs.LG cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

The ability to learn good representations of states is essential for solving large reinforcement learning problems, where exploration, generalization, and transfer are particularly challenging. The Laplacian representation is a promising approach to address these problems by inducing informative state encoding and intrinsic rewards for temporally-extended action discovery and reward shaping. To obtain the Laplacian representation one needs to compute the eigensystem of the graph Laplacian, which is often approximated through optimization objectives compatible with deep learning approaches. These approximations, however, depend on hyperparameters that are impossible to tune efficiently, converge to arbitrary rotations of the desired eigenvectors, and are unable to accurately recover the corresponding eigenvalues. In this paper we introduce a theoretically sound objective and corresponding optimization algorithm for approximating the Laplacian representation. Our approach naturally recovers both the true eigenvectors and eigenvalues while eliminating the hyperparameter dependence of previous approximations. We provide theoretical guarantees for our method and we show that those results translate empirically into robust learning across multiple environments.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 14 of its 23 references that are in the map. First posted on arXiv 16 Oct 2023.

Builds on

14 of 23 references.

References · 14

  1. 2017A Laplacian Framework for Option Discovery in Reinforcement Learningcited by 108
  2. 2017Eigenoption Discovery through the Deep Successor Representationcited by 85
  3. 2018The Laplacian in RL: Learning Representations with Efficient Approximationscited by 83
  4. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 79
  5. 2021Temporal Abstraction in Reinforcement Learning with the Successor Representationcited by 52
  6. 2018On Gradient-Based Learning in Continuous Gamescited by 46
  7. 2023Proto-Value Networks: Scaling Representation Learning with Auxiliary Taskscited by 41
  8. 2018Spectral Inference Networks: Unifying Deep and Spectral Learningcited by 34
  9. 2019Discovering Options for Exploration by Minimizing Cover Timecited by 30
  10. 2022On the Generalization of Representations in Reinforcement Learningcited by 28
  11. 2023Deep Laplacian-based Options for Temporally-Extended Explorationcited by 28
  12. 2021Towards Better Laplacian Representation in Reinforcement Learning with Generalized Graph Drawingcited by 22
2 more
  1. 2022Reachability-Aware Laplacian Representation in Reinforcement Learningcited by 10
  2. 2022No More Pesky Hyperparameters: Offline Hyperparameter Tuning for RLcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 6.

9 papers.

  1. 2025A Survey of State Representation Learning for Deep Reinforcement Learningcited by 17
  2. 2026Laplacian Representations for Decision-Time Planningcited by 4
  3. 2025Reward-Aware Proto-Representations in Reinforcement Learningcited by 3
  4. 2026Exploration and Online Transfer with Behavioral Foundation Modelscited by 1
  5. 2025Revisiting Orbital Minimization Method for Neural Operator Decompositioncited by 0
  6. 2026Intrinsic Reward Policy Optimization for Sparse-Reward Environmentscited by 0
  7. 2026DROGO: Default Representation Objective via Graph Optimization in Reinforcement Learningcited by 0
  8. 2026The Laplacian Keyboard: Beyond the Linear Spancited by 0
  9. 2026Spectral Prioritized Sweeping in Nonstationary Reinforcement Learningcited by 0