mlsanityData to 14 Oct 2026

Wasserstein Barycenter Soft Actor-Critic

arXiv 2506.10167 ↗First posted 11 Jun 2025cs.LG cs.SY eess.SYcited by 1

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Deep off-policy actor-critic algorithms have emerged as the leading framework for reinforcement learning in continuous control domains. However, most of these algorithms suffer from poor sample efficiency, especially in environments with sparse rewards. In this paper, we take a step towards addressing this issue by providing a principled directed exploration strategy. We propose Wasserstein Barycenter Soft Actor-Critic (WBSAC) algorithm, which benefits from a pessimistic actor for temporal difference learning and an optimistic actor to promote exploration. This is achieved by using the Wasserstein barycenter of the pessimistic and optimistic policies as the exploration policy and adjusting the degree of exploration throughout the learning process. We compare WBSAC with state-of-the-art off-policy actor-critic algorithms and show that WBSAC is more sample-efficient on MuJoCo continuous control tasks.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 28 of its 43 references that are in the map. First posted on arXiv 11 Jun 2025.

Builds on

28 of 43 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Wasserstein Adaptive Value Estimation for Actor-Critic Reinforcement Learning17 Jan
  2. 2024Double Actor-Critic with TD Error-Driven Regularization in Reinforcement Learning28 Sep

Other references · 26

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2016OpenAI Gymcited by 1,736
  5. 2018DeepMind Control Suitecited by 956
  6. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  7. 2020SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learningcited by 152
  8. 2021Recent Advances in Reinforcement Learning in Financecited by 110
  9. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous controlcited by 108
  10. 2019Better Exploration with Optimistic Actor-Criticcited by 88
  11. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 58
  12. 2021Tactical Optimism and Pessimism for Deep Reinforcement Learningcited by 55
14 more
  1. 2020Softmax Deep Double Deterministic Policy Gradientscited by 32
  2. 2021Probabilistic Mixture-of-Experts for Efficient Deep Reinforcement Learningcited by 27
  3. 2021Efficient Continuous Control with Double Actors and Regularized Criticscited by 21
  4. 2021Decoupled Exploration and Exploitation Policies for Sample-Efficient Reinforcement Learningcited by 14
  5. 2018ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Searchcited by 12
  6. 2023Optimal Transport Perturbations for Safe Reinforcement Learning with Robustness Guaranteescited by 9
  7. 2021Continuous Control With Ensemble Deep Deterministic Policy Gradientscited by 8
  8. 2023Risk-Aware Reinforcement Learning through Optimal Transport Theorycited by 8
  9. 2023Understanding Reward Ambiguity Through Optimal Transport Theory in Inverse Reinforcement Learningcited by 7
  10. 2021Value Activation for Bias Alleviation: Generalized-activated Deep Double Deterministic Policy Gradientscited by 6
  11. 2024The Synergy Between Optimal Transport Theory and Multi-Agent Reinforcement Learningcited by 6
  12. 2023Wasserstein Actor-Critic: Directed Exploration via Optimism for Continuous-Actions Controlcited by 5
  13. 2024Optimal Transport-Assisted Risk-Sensitive Q-Learningcited by 4
  14. 2023Efficient Reinforcement Learning via Decoupling Exploration and Utilizationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Can Optimal Transport Improve Federated Inverse Reinforcement Learning?cited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Wasserstein Policy Optimization
  2. 2024Improving Actor-Critic Training with Steerable Action-Value Approximation Errors
  3. 2025Langevin Soft Actor-Critic: Efficient Exploration through Uncertainty-Driven Critic Learning
  4. 2026Actor-Critic with Active Importance Sampling
  5. 2024Frugal Actor-Critic: Sample Efficient Off-Policy Deep Reinforcement Learning Using Unique Experiences
  6. 2025Effective Reinforcement Learning Control using Conservative Soft Actor-Critic
  7. 2025Actor-Free Continuous Control via Structurally Maximizable Q-Functions
  8. 2024AFU: Actor-Free critic Updates in off-policy RL for continuous control