# Value function approximation via low-rank models

arXiv: https://arxiv.org/abs/1509.00061 · HTML: https://mlsanity.com/paper/1509.00061/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Aug 2015.

## Built on by (most cited first, 7 of 7)

- [Tensor and Matrix Low-Rank Value-Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/2201.09736/) (2022)
- [Low-rank State-action Value-function Approximation](https://mlsanity.com/paper/2104.08805/) (2021)
- [Matrix Low-Rank Trust Region Policy Optimization](https://mlsanity.com/paper/2405.17625/) (2024)
- [Matrix Low-Rank Approximation For Policy Gradient Methods](https://mlsanity.com/paper/2405.17626/) (2024)
- [Detection and Evaluation of Clusters within Sequential Data](https://mlsanity.com/paper/2210.01679/) (2022)
- [Reinforcement Learning for Resilient Power Grids](https://mlsanity.com/paper/2212.04069/) (2022)
- [Uncertainty-aware Low-Rank Q-Matrix Estimation for Deep Reinforcement Learning](https://mlsanity.com/paper/2111.10103/) (2021)

## Abstract (arXiv)

We propose a novel value function approximation technique for Markov decision processes. We consider the problem of compactly representing the state-action value function using a low-rank and sparse matrix model. The problem is to decompose a matrix that encodes the true value function into low-rank and sparse components, and we achieve this using Robust Principal Component Analysis (PCA). Under minimal assumptions, this Robust PCA problem can be solved exactly via the Principal Component Pursuit convex optimization problem. We experiment the procedure on several examples and demonstrate that our method yields approximations essentially identical to the true function.