mlsanityData to 14 Oct 2026

Factorized Spectral Representations for Reinforcement Learning

arXiv 2607.13498 ↗First posted 15 Jul 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Learning a compact model of the world from interaction data is central to sample-efficient deep reinforcement learning. Spectral representation methods have become the leading paradigm for representation learning in continuous control by taking a matrix view of the transition kernel, with state-action pairs on one side and next states on the other, and learning a low-rank factorization through self-supervised contrastive objectives. We take this view one step further. The transition kernel is naturally a three-mode tensor over states, actions, and next states, and a CP decomposition gives one feature map per mode. We propose FaStR, which fits this decomposition with a noise contrastive objective, producing separate state, action, and next-state encoders that together form a single spectral representation. The factored form yields a smaller hypothesis class, and the sample size needed for representation learning shrinks by a factor that scales with the smaller of the state and action dimensions. Empirically, FaStR delivers its largest gains on high-dimensional locomotion tasks whose dynamics align with the factored structure, and the learned state encoder transfers intact across actuator shift while only the action encoder is retrained.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 28 of its 39 references that are in the map. First posted on arXiv 15 Jul 2026.

Builds on

28 of 39 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Spectral Representation-based Reinforcement Learning17 Dec
  2. 2025Shift Before You Learn: Enabling Low-Rank Representations in Reinforcement Learning5 Sep

Other references · 26

  1. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  6. 2016HyperNetworkscited by 1,027
  7. 2018DeepMind Control Suitecited by 956
  8. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 520
  9. 2020FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPscited by 233
  10. 2019Reinforcement Learning in Feature Space: Matrix Bandit, Kernels, and Regret Boundcited by 201
  11. 2021Bilinear Classes: A Structural Framework for Provable Generalization in RLcited by 178
  12. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
14 more
  1. 2021Representation Learning for Online and Offline RL in Low-rank MDPscited by 129
  2. 2018Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiencycited by 105
  3. 2021Model-free Representation Learning and Exploration in Low-rank MDPscited by 82
  4. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 75
  5. 2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 57
  6. 2022Making Linear MDPs Practical via Contrastive Representation Learningcited by 52
  7. 2022Spectral Decomposition Representation for Reinforcement Learningcited by 36
  8. 2021Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learningcited by 25
  9. 2022Tensor and Matrix Low-Rank Value-Function Approximation in Reinforcement Learningcited by 16
  10. 2024Diffusion Spectral Representation for Reinforcement Learningcited by 15
  11. 2023Spectral Entry-wise Matrix Estimation for Low-Rank Reinforcement Learningcited by 10
  12. 2025Spectral Bellman Method: Unifying Representation and Exploration in RLcited by 4
  13. 2024BECAUSE: Bilinear Causal Representation for Generalizable Offline Model-based Reinforcement Learningcited by 3
  14. 2025Multilinear Tensor Low-Rank Approximation for Policy-Gradient Methods in Reinforcement Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025From Pixels to Factors: Learning Independently Controllable State Variables for Reinforcement Learning
  2. 2024Towards Principled Representation Learning from Videos for Reinforcement Learning
  3. 2026Deep Spectral Learning of Embedded Latent Transfer Operators for Stochastic Dynamical Systems
  4. 2025A Primer on SO(3) Action Representations in Deep Reinforcement Learning
  5. 2026Matrix-Space Reinforcement Learning for Reusing Local Transition Geometry
  6. 2022Reinforcement Learning with Action-Free Pre-Training from Videos
  7. 2024Pre-trained Visual Dynamics Representations for Efficient Policy Learning
  8. 2026Can We Really Learn One Representation to Optimize All Rewards?