mlsanityData to 7 Oct 2026

Reachability-Aware Laplacian Representation in Reinforcement Learning

arXiv 2210.13153 ↗First posted 24 Oct 2022cs.LG cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

In Reinforcement Learning (RL), Laplacian Representation (LapRep) is a task-agnostic state representation that encodes the geometry of the environment. A desirable property of LapRep stated in prior works is that the Euclidean distance in the LapRep space roughly reflects the reachability between states, which motivates the usage of this distance for reward shaping. However, we find that LapRep does not necessarily have this property in general: two states having small distance under LapRep can actually be far away in the environment. Such mismatch would impede the learning process in reward shaping. To fix this issue, we introduce a Reachability-Aware Laplacian Representation (RA-LapRep), by properly scaling each dimension of LapRep. Despite the simplicity, we demonstrate that RA-LapRep can better capture the inter-state reachability as compared to LapRep, through both theoretical explanations and experimental results. Additionally, we show that this improvement yields a significant boost in reward shaping performance and also benefits bottleneck state discovery.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 20 of its 42 references that are in the map. First posted on arXiv 24 Oct 2022.

Builds on

20 of 42 references.

References · 20

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  5. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  6. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  7. 2018Count-Based Exploration with the Successor Representationcited by 112
  8. 2018Episodic Curiosity through Reachabilitycited by 109
  9. 2017A Laplacian Framework for Option Discovery in Reinforcement Learningcited by 108
  10. 2017Eigenoption Discovery through the Deep Successor Representationcited by 85
  11. 2018The Laplacian in RL: Learning Representations with Efficient Approximationscited by 83
  12. 2019Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discoverycited by 65
8 more
  1. 2020Generating Adjacency-Constrained Subgoals in Hierarchical Reinforcement Learningcited by 55
  2. 2021Temporal Abstraction in Reinforcement Learning with the Successor Representationcited by 52
  3. 2018Investigating Human Priors for Playing Video Gamescited by 39
  4. 2018Decoupling Dynamics and Reward for Transfer Learningcited by 23
  5. 2021Towards Better Laplacian Representation in Reinforcement Learning with Generalized Graph Drawingcited by 22
  6. 2022Temporal Abstractions-Augmented Temporally Contrastive Learning: An Alternative to the Laplacian in RLcited by 6
  7. 2013A New Notion of Effective Resistance for Directed Graphs-Part I: Definition and Propertiescited by 4
  8. 2018Effective Resistance Preserving Directed Graph Symmetrizationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 5; 2026: 4.

10 papers.

  1. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Modelscited by 77
  2. 2025A Survey of State Representation Learning for Deep Reinforcement Learningcited by 17
  3. 2025Episodic Novelty Through Temporal Distancecited by 15
  4. 2023Proper Laplacian Representation Learningcited by 9
  5. 2026Laplacian Representations for Decision-Time Planningcited by 4
  6. 2025Reward-Aware Proto-Representations in Reinforcement Learningcited by 3
  7. 2026Impact of Connectivity on Laplacian Representations in Reinforcement Learningcited by 1
  8. 2025Learning The Minimum Action Distancecited by 0
  9. 2026DROGO: Default Representation Objective via Graph Optimization in Reinforcement Learningcited by 0
  10. 2026Learning Commute-Time-Preserving World Models for Planningcited by 0