mlsanityData to 7 Oct 2026

Boosting Soft Q-Learning by Bounding

arXiv 2406.18033 ↗First posted 26 Jun 2024cs.LG cs.AI stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

An agent's ability to leverage past experience is critical for efficiently solving new tasks. Prior work has focused on using value function estimates to obtain zero-shot approximations for solutions to a new task. In soft Q-learning, we show how any value function estimate can also be used to derive double-sided bounds on the optimal value function. The derived bounds lead to new approaches for boosting training performance which we validate experimentally. Notably, we find that the proposed framework suggests an alternative method for updating the Q-function, leading to boosted performance.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 17 of its 27 references that are in the map. First posted on arXiv 26 Jun 2024.

Builds on

17 of 27 references.

References · 17

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2016Progressive Neural Networkscited by 1,332
  3. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  4. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  5. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
  6. 2020SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learningcited by 150
  7. 2022Deep Hierarchical Planning from Pixelscited by 96
  8. 2018Composable Deep Reinforcement Learning for Robotic Manipulationcited by 70
  9. 2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 63
  10. 2021Identifiability in inverse reinforcement learningcited by 55
  11. 2017Global optimization of Lipschitz functionscited by 39
  12. 2016Learning to Play in a Day: Faster Deep Reinforcement Learning by Optimality Tighteningcited by 21
5 more
  1. 2021Constructing a Good Behavior Basis for Transfer using Generalized Policy Updatescited by 14
  2. 2020Self-Imitation Learning via Generalized Lower Bound Q-learningcited by 13
  3. 2022Utilizing Prior Solutions for Reward Shaping and Composition in Entropy-Regularized Reinforcement Learningcited by 8
  4. 2020Qgraph-bounded Q-learning: Stabilizing Model-Free Off-Policy Deep Reinforcement Learningcited by 3
  5. 2023Bounding the Optimal Value Function in Compositional Reinforcement Learningcited by 3

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.

  1. 2025Bootstrapped Reward Shapingcited by 3
  2. 2025Causality-informed Anomaly Detection in Partially Observable Sensor Networks: Moving beyond Correlationscited by 2
  3. 2024Efficient Imitation Without Demonstrations via Value-Penalized Auxiliary Control from Examplescited by 1
  4. 2025Provably Efficient Reward Transfer in Reinforcement Learning with Discrete Markov Decision Processescited by 0