mlsanityData to 7 Oct 2026

Logistic Q-Learning

arXiv 2010.11151 ↗First posted 21 Oct 2020cs.LG cs.AI stat.MLcited by 34

Not on the map. Why?

Abstract (arXiv)

We propose a new reinforcement learning algorithm derived from a regularized linear-programming formulation of optimal control in MDPs. The method is closely related to the classic Relative Entropy Policy Search (REPS) algorithm of Peters et al. (2010), with the key difference that our method introduces a Q-function that enables efficient exact model-free implementation. The main feature of our algorithm (called QREPS) is a convex loss function for policy evaluation that serves as a theoretically sound alternative to the widely used squared Bellman error. We provide a practical saddle-point optimization method for minimizing this loss function and provide an error-propagation analysis that relates the quality of the individual updates to the performance of the output policy. Finally, we demonstrate the effectiveness of our method on a range of benchmark problems.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 10; 2023: 10; 2024: 3; 2025: 4; 2026: 3.

34 papers.

Sort:

By area:

  1. 2023Extreme Q-Learning: MaxEnt RL without Entropycited by 113
  2. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  3. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
  4. 2023Dual RL: Unification and New Methods for Reinforcement and Imitation Learningcited by 48
  5. 2022Why Should I Trust You, Bellman? The Bellman Error is a Poor Replacement for Value Errorcited by 31
  6. 2022Proximal Point Imitation Learningcited by 25
  7. 2022From Dirichlet to Rubin: Optimistic Exploration in RL without Bonusescited by 22
  8. 2022Your Policy Regularizer is Secretly an Adversarycited by 21
  9. 2023Coherent Soft Imitation Learningcited by 21
  10. 2023ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for Last-Iterate Convergence in Constrained MDPscited by 19
  11. 2023Offline Primal-Dual Reinforcement Learning for Linear MDPscited by 17
  12. 2023TD Convergence: An Optimization Perspectivecited by 15
22 more
  1. 2021Efficient Performance Bounds for Primal-Dual Reinforcement Learning from Demonstrationscited by 9
  2. 2022Efficient Global Planning in Large MDPs via Stochastic Primal-Dual Optimizationcited by 8
  3. 2024The Central Role of the Loss Function in Reinforcement Learningcited by 8
  4. 2023Stability of Q-Learning Through Design and Optimismcited by 7
  5. 2021Globally Optimal Hierarchical Reinforcement Learning for Linearly-Solvable Markov Decision Processescited by 6
  6. 2023Convex Q Learning in a Stochastic Environment: Extended Versioncited by 5
  7. 2025Provably Efficient RL under Episode-Wise Safety in Constrained MDPs with Linear Function Approximationcited by 4
  8. 2023Parameterized Projected Bellman Operatorcited by 3
  9. 2025Multi-Step Alignment as Markov Games: An Optimistic Online Gradient Descent Approach with Convergence Guaranteescited by 3
  10. 2024Convergence of a model-free entropy-regularized inverse reinforcement learning algorithmcited by 2
  11. 2024Offline RL via Feature-Occupancy Gradient Ascentcited by 2
  12. 2021Near Optimal Policy Optimization via REPScited by 1
  13. 2022Sufficient Exploration for Convex Q-learningcited by 1
  14. 2025Analysis of approximate linear programming solution to Markov decision problem with log barrier functioncited by 1
  15. 2022Marginalized Operators for Off-policy Reinforcement Learningcited by 0
  16. 2022A Parametric Class of Approximate Gradient Updates for Policy Optimizationcited by 0
  17. 2022Model-Free Characterizations of the Hamilton-Jacobi-Bellman Equation and Convex Q-Learning in Continuous Timecited by 0
  18. 2023LLQL: Logistic Likelihood Q-Learning for Reinforcement Learningcited by 0
  19. 2025Priors Matter: Addressing Misspecification in Bayesian Deep Q-Learningcited by 0
  20. 2026Generative Modeling by Value-Driven Transportcited by 0
  21. 2026Online Policy Evaluation for MDPs with Dynamic UBSR Measurescited by 0
  22. 2026Lifted Bellman Linear Programming for Offline Reinforcement Learningcited by 0