mlsanityData to 7 Oct 2026

Learning Good State and Action Representations via Tensor Decomposition

arXiv 2105.01136 ↗First posted 3 May 2021stat.ML cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

The transition kernel of a continuous-state-action Markov decision process (MDP) admits a natural tensor structure. This paper proposes a tensor-inspired unsupervised learning method to identify meaningful low-dimensional state and action representations from empirical trajectories. The method exploits the MDP's tensor structure by kernelization, importance sampling and low-Tucker-rank approximation. This method can be further used to cluster states and actions respectively and find the best discrete MDP abstraction. We provide sharp statistical error bounds for tensor concentration and the preservation of diffusion distance after embedding. We further prove that the learned state/action abstractions provide accurate approximations to latent block structures if they exist, enabling function approximation in downstream tasks such as policy evaluation.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 26 of its 65 references that are in the map. First posted on arXiv 3 May 2021.

Builds on

26 of 65 references.

References · 26

  1. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
  2. 2016Contextual Decision Processes with Low Bellman Rank are PAC-Learnablecited by 290
  3. 2020FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPscited by 232
  4. 2016Tensor Decomposition for Signal Processing and Machine Learningcited by 223
  5. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
  6. 2019Is a Good Representation Sufficient for Sample Efficient Reinforcement Learning?cited by 140
  7. 2019Learning with Good Feature Representations in Bandits and in RL with a Generative Modelcited by 124
  8. 2019Kinematic State Abstraction and Provably Efficient Rich-Observation Reinforcement Learningcited by 114
  9. 2021Model-free Representation Learning and Exploration in Low-rank MDPscited by 82
  10. 2014A statistical model for tensor PCAcited by 72
  11. 2017Tensor SVD: Statistical and Computational Limitscited by 52
  12. 2017Eigendecompositions of Transfer Operators in Reproducing Kernel Hilbert Spacescited by 48
14 more
  1. 2016Rate-Optimal Perturbation Bounds for Singular Subspaces with Applications to High-Dimensional Statisticscited by 45
  2. 2020An Optimal Statistical and Computational Framework for Generalized Tensor Estimationcited by 39
  3. 2018Generalized Canonical Polyadic Tensor Decompositioncited by 34
  4. 2019On Online Learning in Kernelized Markov Decision Processescited by 32
  5. 2014Guaranteed Non-Orthogonal Tensor Decomposition via Alternating Rank-1 Updatescited by 21
  6. 2018Spectral State Compression of Markov Processescited by 18
  7. 2015Provable Sparse Tensor Decompositioncited by 16
  8. 2020Spectral Learning on Matrices and Tensorscited by 16
  9. 2016Reinforcement Learning in Rich-Observation MDPs using Spectral Methodscited by 13
  10. 2019Learning Markov models via low-rank optimizationcited by 12
  11. 2016Cross: Efficient Low-rank Tensor Completioncited by 10
  12. 2018Optimal Sparse Singular Value Decomposition for High-dimensional High-order Datacited by 6
  13. 2019Learning low-dimensional state embeddings and metastable clusters from time series datacited by 6
  14. 2018Spectral thresholding for the estimation of Markov chain transition operatorscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 1; 2024: 1; 2025: 1; 2026: 1.

7 papers.

  1. 2021Representation Learning for Online and Offline RL in Low-rank MDPscited by 128
  2. 2022Learning Bellman Complete Representations for Offline Policy Evaluationcited by 17
  3. 2022Overcoming the Long Horizon Barrier for Sample-Efficient Reinforcement Learning with Latent Low-Rank Structurecited by 7
  4. 2024Off-policy Evaluation with Deeply-abstracted Statescited by 2
  5. 2025Addressing Finite-Horizon MDPs via Low-Rank Tensor Value Approximationcited by 1
  6. 2026Realistic Curriculum Reinforcement Learning for Autonomous and Sustainable Marine Vessel Navigationcited by 1
  7. 2023Safe Reinforcement Learning in Tensor Reproducing Kernel Hilbert Spacecited by 0