mlsanityData to 14 Oct 2026

On the connection between Bregman divergence and value in regularized Markov decision processes

arXiv 2210.12160 ↗First posted 21 Oct 2022cs.LG cs.AI math.OCcited by 2

Not on the map. Why?

Abstract (arXiv)

In this short note we derive a relationship between the Bregman divergence from the current policy to the optimal policy and the suboptimality of the current value function in a regularized Markov decision process. This result has implications for multi-task reinforcement learning, offline reinforcement learning, and regret analysis under function approximation, among others.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 2 of its 4 references that are in the map. First posted on arXiv 21 Oct 2022.

Builds on

2 of 4 references.

References · 2

  1. 2020On the Global Convergence Rates of Softmax Policy Gradient Methodscited by 264
  2. 2020Making Sense of Reinforcement Learning and Probabilistic Inferencecited by 30

Built on by

Cited by year

Citing papers per year: 2023: 2.

2 papers.

  1. 2023Probabilistic Inference in Reinforcement Learning Done Rightcited by 11
  2. 2023Efficient Exploration via Epistemic-Risk-Seeking Policy Optimizationcited by 10

Similar, not cited

Close in wording, no citation either way

  1. 2024Optimal Rates of Convergence for Entropy Regularization in Discounted Markov Decision Processes
  2. 2026Mathematical methods of reinforcement learning
  3. 2026Randomized Optimal Switching Problem and Related Mirror Descent Flow
  4. 2026Reinforcement Learning for Exponential Utility: Algorithms and Convergence in Discounted MDPs
  5. 2026f-Divergence Regularized RLHF: Two Tales of Sampling and Unified Analyses
  6. 2024Reinforcement Learning for Infinite-Horizon Average-Reward Linear MDPs via Approximation by Discounted-Reward MDPs
  7. 2026Value Mirror Descent for Reinforcement Learning
  8. 2024Robust Offline Reinforcement Learning with Linearly Structured f-Divergence Regularization