mlsanityData to 7 Oct 2026

DSAC-C: Constrained Maximum Entropy for Robust Discrete Soft-Actor Critic

arXiv 2310.17173 ↗First posted 26 Oct 2023cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We present a novel extension to the family of Soft Actor-Critic (SAC) algorithms. We argue that based on the Maximum Entropy Principle, discrete SAC can be further improved via additional statistical constraints derived from a surrogate critic policy. Furthermore, our findings suggests that these constraints provide an added robustness against potential domain shifts, which are essential for safe deployment of reinforcement learning agents in the real-world. We provide theoretical analysis and show empirical results on low data regimes for both in-distribution and out-of-distribution variants of Atari 2600 games.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 33 of its 50 references that are in the map. First posted on arXiv 26 Oct 2023.

Builds on

33 of 50 references.

References · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2015Trust Region Policy Optimizationcited by 2,503
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  7. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  8. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  9. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  10. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  11. 2015Prioritized Experience Replaycited by 1,070
  12. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
21 more
  1. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  2. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  3. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  4. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  5. 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 461
  6. 2020Calibrating Deep Neural Networks using Focal Losscited by 353
  7. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  8. 2017Noisy Networks for Explorationcited by 262
  9. 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 218
  10. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
  11. 2019Soft Actor-Critic for Discrete Action Settingscited by 161
  12. 2017Parameter Space Noise for Explorationcited by 158
  13. 2019IPO: Interior-point Policy Optimization under Constraintscited by 119
  14. 2020Learning to Walk in the Real World with Minimal Human Effortcited by 103
  15. 2020Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Researchcited by 78
  16. 2020Robust Deep Reinforcement Learning through Adversarial Losscited by 73
  17. 2019Soft Policy Gradient Method for Maximum Entropy Deep Reinforcement Learningcited by 18
  18. 2019Uncertainty-Based Out-of-Distribution Detection in Deep Reinforcement Learningcited by 15
  19. 2020Meta-SAC: Auto-tune the Entropy Temperature of Soft Actor-Critic via Metagradientcited by 14
  20. 2022Revisiting Discrete Soft Actor-Criticcited by 11
  21. 2021Target Entropy Annealing for Discrete Soft Actor-Criticcited by 10

Built on by

Not cited yet.