Spectral Alignment in Forward-Backward Representations via Temporal Abstraction
Not on the map. Why?
Abstract (arXiv)
Forward-backward (FB) representations provide a powerful framework for learning the successor representation (SR) in continuous spaces by enforcing a low-rank factorization. However, a fundamental spectral mismatch often exists between the high-rank transition dynamics of continuous environments and the low-rank bottleneck of the FB architecture, making accurate low-rank representation learning difficult. In this work, we analyze temporal abstraction as a mechanism to mitigate this mismatch. By characterizing the spectral properties of the transition operator, we show that temporal abstraction acts analogously to a low-pass filter that suppresses high-frequency spectral components. This suppression reduces the effective rank of the induced SR while preserving a formal bound on the resulting value function error. Empirically, we show that this alignment is a key factor for stable FB learning, particularly at high discount factors where bootstrapping becomes error-prone. Our results identify temporal abstraction as a principled mechanism for shaping the spectral structure of the underlying MDP and enabling effective long-horizon representations in continuous control.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 34 references that are in the map. First posted on arXiv 20 Mar 2026.
Builds on
19 of 34 references.
References · 19
- 2016Successor Features for Transfer in Reinforcement Learningcited by 331
- 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 216
- 2021Learning One Representation to Optimize All Rewardscited by 110
- 2017A Laplacian Framework for Option Discovery in Reinforcement Learningcited by 108
- 2025Reinforcement Learning with Action Chunkingcited by 97
- 2017Eigenoption Discovery through the Deep Successor Representationcited by 85
- 2022Does Zero-Shot Reinforcement Learning Exist?cited by 79
- 2016Deep Successor Reinforcement Learningcited by 72
- 2021Learning Successor States and Goal-Dependent Values: A Mathematical Viewpointcited by 67
- 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 51
- 2016Deep Reinforcement Learning with Successor Features for Navigation across Similar Environmentscited by 50
- 2021TempoRL: Learning When to Actcited by 25
7 more
- 2025Fast Adaptation with Behavioral Foundation Modelscited by 20
- 2025Epistemically-guided forward-backward explorationcited by 6
- 2025Shift Before You Learn: Enabling Low-Rank Representations in Reinforcement Learningcited by 4
- 2026Laplacian Representations for Decision-Time Planningcited by 4
- 2025SR-Reward: Taking The Path More Traveledcited by 3
- 2023On Double Descent in Reinforcement Learning with LSTD and Random Featurescited by 2
- 2022Temporally Extended Successor Representationscited by 1
Built on by
Not cited yet.