mlsanityData to 14 Oct 2026

Exponential Family Model-Based Reinforcement Learning via Score Matching

arXiv 2112.14195 ↗First posted 28 Dec 2021cs.LG stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

We propose an optimistic model-based algorithm, dubbed SMRL, for finite-horizon episodic reinforcement learning (RL) when the transition model is specified by exponential family distributions with d parameters and the reward is bounded and known. SMRL uses score matching, an unnormalized density estimation technique that enables efficient estimation of the model parameter by ridge regression. Under standard regularity assumptions, SMRL achieves tilde O(d√H3T) online regret, where H is the length of each episode and T is the total number of interactions (ignoring polynomial dependence on structural scale parameters).

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 31 of its 64 references that are in the map. First posted on arXiv 28 Dec 2021.

Builds on

31 of 64 references.

References · 31

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 520
  3. 2017Minimax Regret Bounds for Reinforcement Learningcited by 503
  4. 2016Contextual Decision Processes with Low Bellman Rank are PAC-Learnablecited by 292
  5. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  6. 2020Model-Based Reinforcement Learning with Value-Targeted Regressioncited by 234
  7. 2021The Statistical Complexity of Interactive Decision Makingcited by 218
  8. 2019Provably Efficient Exploration in Policy Optimizationcited by 205
  9. 2019Reinforcement Learning in Feature Space: Matrix Bandit, Kernels, and Regret Boundcited by 201
  10. 2020Learning Near Optimal Policies with Low Inherent Bellman Errorcited by 181
  11. 2017Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learningcited by 176
  12. 2017Deep Exploration via Randomized Value Functionscited by 156
19 more
  1. 2019Learning with Good Feature Representations in Bandits and in RL with a Generative Modelcited by 127
  2. 2014Model-based Reinforcement Learning and the Eluder Dimensioncited by 116
  3. 2020Naive Exploration is Optimal for Online LQRcited by 116
  4. 2014Generalization and Exploration via Randomized Value Functionscited by 112
  5. 2019Frequentist Regret Bounds for Randomized Least-Squares Value Iterationcited by 101
  6. 2019Sample Complexity of Reinforcement Learning using Linearly Combined Model Ensemblescited by 100
  7. 2020Information Theoretic Regret Bounds for Online Nonlinear Controlcited by 90
  8. 2018Regret Bounds for Robust Adaptive Control of the Linear Quadratic Regulatorcited by 86
  9. 2020Active Learning for Nonlinear System Identification with Guaranteescited by 80
  10. 2019Learning Linear-Quadratic Regulators Efficiently with only √T Regretcited by 70
  11. 2019Worst-Case Regret Bounds for Exploration via Randomized Value Functionscited by 57
  12. 2013Density Estimation in Infinite Dimensional Exponential Familiescited by 42
  13. 2019An Online Learning Approach to Model Predictive Controlcited by 30
  14. 2019Exponential Family Estimation via Adversarial Dynamics Embeddingcited by 30
  15. 2017Posterior sampling for reinforcement learning: worst-case regret boundscited by 29
  16. 2020Improved Worst-Case Regret Bounds for Randomized Least-Squares Value Iterationcited by 19
  17. 2019Introducing an Explicit Symplectic Integration Scheme for Riemannian Manifold Hamiltonian Monte Carlocited by 10
  18. 2017Efficient and principled score estimation with Nystr"om kernel exponential familiescited by 9
  19. 2017Kernel Conditional Exponential Familycited by 7

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 2; 2025: 1.

5 papers.

  1. 2023Learning a Diffusion Model Policy from Rewards via Q-Score Matchingcited by 138
  2. 2024Sampling from Energy-based Policies using Diffusioncited by 17
  3. 2022Bilinear Exponential Family of MDPs: Frequentist Regret Bound with Tractable Exploration and Planningcited by 12
  4. 2024Provably Efficient Reinforcement Learning with Multinomial Logit Function Approximationcited by 10
  5. 2025Episodic Contextual Bandits with Knapsacks under Conversion Modelscited by 1