mlsanityData to 7 Oct 2026

Low-rank State-action Value-function Approximation

arXiv 2104.08805 ↗First posted 18 Apr 2021cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

Value functions are central to Dynamic Programming and Reinforcement Learning but their exact estimation suffers from the curse of dimensionality, challenging the development of practical value-function (VF) estimation algorithms. Several approaches have been proposed to overcome this issue, from non-parametric schemes that aggregate states or actions to parametric approximations of state and action VFs via, e.g., linear estimators or deep neural networks. Relevantly, several high-dimensional state problems can be well-approximated by an intrinsic low-rank structure. Motivated by this and leveraging results from low-rank optimization, this paper proposes different stochastic algorithms to estimate a low-rank factorization of the Q(s, a) matrix. This is a non-parametric alternative to VF approximation that dramatically reduces the computational and sample complexities relative to classical Q-learning methods that estimate Q(s,a) separately for each state-action pair.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 9 of its 21 references that are in the map. First posted on arXiv 18 Apr 2021.

Builds on

9 of 21 references.

References · 9

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2016OpenAI Gymcited by 1,684
  3. 2016Contextual Decision Processes with Low Bellman Rank are PAC-Learnablecited by 290
  4. 2014Generalized Low Rank Modelscited by 55
  5. 2017Revisiting Normalized Gradient Descent: Fast Evasion of Saddle Pointscited by 28
  6. 2020Sample Efficient Reinforcement Learning via Low-Rank Matrix Estimationcited by 25
  7. 2019Harnessing Structures for Value-Based Planning and Reinforcement Learningcited by 24
  8. 2018Factorized Q-Learning for Large-Scale Multi-Agent Systemscited by 19
  9. 2015Value function approximation via low-rank modelscited by 7

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 0; 2024: 3; 2025: 3.

10 papers.

  1. 2022Tensor and Matrix Low-Rank Value-Function Approximation in Reinforcement Learningcited by 16
  2. 2022Overcoming the Long Horizon Barrier for Sample-Efficient Reinforcement Learning with Latent Low-Rank Structurecited by 7
  3. 2022Towards Global Optimality in Cooperative MARL with the Transformation And Distillation Frameworkcited by 5
  4. 2024Tensor Low-rank Approximation of Finite-horizon Value Functionscited by 3
  5. 2025Multilinear Tensor Low-Rank Approximation for Policy-Gradient Methods in Reinforcement Learningcited by 3
  6. 2022Applications of Reinforcement Learning in Finance -- Trading with a Double Deep Q-Networkcited by 2
  7. 2024Matrix Low-Rank Trust Region Policy Optimizationcited by 2
  8. 2024Matrix Low-Rank Approximation For Policy Gradient Methodscited by 2
  9. 2025Addressing Finite-Horizon MDPs via Low-Rank Tensor Value Approximationcited by 1
  10. 2025ADARL: Adaptive Low-Rank Structures for Robust Policy Learning under Uncertaintycited by 0