Data to 7 Oct 2026

FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPs

arXiv 2006.10814 ↗First posted 18 Jun 2020cs.LG stat.MLcited by 232

Not on the map. Why?

Cited by 40 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In order to deal with the curse of dimensionality in reinforcement learning (RL), it is common practice to make parametric assumptions where values or policies are functions of some low dimensional feature space. This work focuses on the representation learning question: how can we learn such features? Under the assumption that the underlying (unknown) dynamics correspond to a low rank transition matrix, we show how the representation learning question is related to a particular non-linear matrix decomposition problem. Structurally, we make precise connections between these low rank MDPs and latent variable models, showing how they significantly generalize prior formulations for representation learning in RL. Algorithmically, we develop FLAMBE, which engages in exploration and representation learning for provably efficient RL in low rank transition models.

Key facts. Cited by 232 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 30; 2022: 51; 2023: 53; 2024: 48; 2025: 30; 2026: 19.

232 papers.

Sort:

By area:

2021Bellman Eluder Dimension: New Rich Classes of RL Problems, and Sample-Efficient Algorithmscited by 222
2021Bilinear Classes: A Structural Framework for Provable Generalization in RLcited by 176
2021Pessimistic Model-based Offline Reinforcement Learning under Partial Coveragecited by 160
2022Hybrid RL: Using Both Offline and Online Data Can Make RL Efficientcited by 158
2021Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learningcited by 157
2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 137
2021Representation Learning for Online and Offline RL in Low-rank MDPscited by 128
2022Offline Reinforcement Learning with Realizability and Single-policy Concentrabilitycited by 122
2023What and How does In-Context Learning Learn? Bayesian Model Averaging, Parameterization, and Generalizationcited by 112
2021Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learningcited by 109
2022When Is Partially Observable Reinforcement Learning Not Scary?cited by 100
2021Model-free Representation Learning and Exploration in Low-rank MDPscited by 82

38 more

2024RLHF from Heterogeneous Feedback via Personalization and Preference Aggregationcited by 71
2023Reinforcement Learning with Human Feedback: Learning Dynamic Choices via Pessimismcited by 69
2022Denoised MDPs: Learning World Models Better Than the World Itselfcited by 66
2024Self-Exploring Language Models: Active Preference Elicitation for Online Alignmentcited by 63
2022Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environmentscited by 54
2022Efficient Reinforcement Learning in Block MDPs: A Model-free Representation Learning Approachcited by 51
2022Exploration via Elliptical Episodic Bonusescited by 51
2022On the Principles of Parsimony and Self-Consistency for the Emergence of Intelligencecited by 50
2022Reward-Free RL is No Harder Than Reward-Aware RL in Linear Markov Decision Processescited by 49
2022Making Linear MDPs Practical via Contrastive Representation Learningcited by 49
2021The Power of Exploiter: Provable Multi-Agent RL in Large State Spacescited by 48
2023Making RL with Preference-based Feedback Efficient via Randomizationcited by 46
2021Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent Statescited by 42
2022PAC Reinforcement Learning for Predictive State Representationscited by 42
2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
2022Provable Reinforcement Learning with a Short-Term Memorycited by 40
2025Towards General-Purpose Model-Free Reinforcement Learningcited by 40
2022Provable Benefits of Representational Transfer in Reinforcement Learningcited by 38
2021The Information Geometry of Unsupervised Reinforcement Learningcited by 36
2022Provably Efficient Reinforcement Learning in Partially Observable Dynamical Systemscited by 35
2022Spectral Decomposition Representation for Reinforcement Learningcited by 35
2021Provable Representation Learning for Imitation with Contrastive Fourier Featurescited by 34
2025LoRe: Personalizing LLMs via Low-Rank Reward Modelingcited by 34
2024Large Language Models as Markov Chainscited by 33
2021Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvaturecited by 32
2023Near-Minimax-Optimal Risk-Sensitive Reinforcement Learning with CVaRcited by 32
2022Model-based RL with Optimistic Posterior Sampling: Structural Conditions and Sample Complexitycited by 31
2022A General Framework for Sample-Efficient Function Approximation in Reinforcement Learningcited by 31
2023Adversarial Model for Offline Reinforcement Learningcited by 31
2021Robust Policy Gradient against Strong Data Corruptioncited by 30
2022Provable Benefit of Multitask Representation Learning in Reinforcement Learningcited by 29
2022Provably Efficient Model-Free Constrained RL with Linear Function Approximationcited by 29
2021Provable RL with Exogenous Distractors via Multistep Inverse Dynamicscited by 28
2021Sample-Efficient Reinforcement Learning Is Feasible for Linearly Realizable MDPs with Limited Revisitingcited by 27
2022On the Statistical Efficiency of Reward-Free Exploration in Non-Linear RLcited by 27
2023The Benefits of Being Distributional: Small-Loss Bounds for Reinforcement Learningcited by 27
2021Interactive Learning from Activity Descriptioncited by 26
2022Sample-Efficient Reinforcement Learning in the Presence of Exogenous Informationcited by 26

50 most cited of 232 All 232 (JSON)