mlsanityData to 14 Oct 2026

MinMaxMin Q-learning

arXiv 2402.05951 ↗First posted 3 Feb 2024cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

MinMaxMin Q-learning is a novel optimistic Actor-Critic algorithm that addresses the problem of overestimation bias (Q-estimations are overestimating the real Q-values) inherent in conservative RL algorithms. Its core formula relies on the disagreement among Q-networks in the form of the min-batch MaxMin Q-networks distance which is added to the Q-target and used as the priority experience replay sampling-rule. We implement MinMaxMin on top of TD3 and TD7, subjecting it to rigorous testing against state-of-the-art continuous-space algorithms-DDPG, TD3, and TD7-across popular MuJoCo and Bullet environments. The results show a consistent performance improvement of MinMaxMin over DDPG, TD3, and TD7 across all tested tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 8 of its 16 references that are in the map. First posted on arXiv 3 Feb 2024.

Builds on

8 of 16 references.

References · 8

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2017Hindsight Experience Replaycited by 1,136
  5. 2015Prioritized Experience Replaycited by 1,099
  6. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 302
  7. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 75
  8. 2022From Dirichlet to Rubin: Optimistic Exploration in RL without Bonusescited by 22

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Moderate Actor-Critic Methods: Controlling Overestimation Bias via Expectile Loss
  2. 2026Revisiting Overestimation Bias Problem of Q-learning: Settling Large Discrete Action Space via Action Intersection
  3. 2025Deep SOR Minimax Q-learning for Two-player Zero-sum Game
  4. 2026Revisiting TD Target Aggregation under Uncertainty in Q-Learning
  5. 2025Efficient Q-Learning and Actor-Critic Methods for Robust Average-Reward Reinforcement Learning
  6. 2025Langevin Soft Actor-Critic: Efficient Exploration through Uncertainty-Driven Critic Learning
  7. 2025Dual Ensembled Multiagent Q-Learning with Hypernet Regularizer
  8. 2026Debiased Model-based Representations for Sample-efficient Continuous Control