# A Strategy-Oriented Bayesian Soft Actor-Critic Model

arXiv: https://arxiv.org/abs/2303.04193 · HTML: https://mlsanity.com/paper/2303.04193/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 25 references that are in the map.
- First posted on arXiv 7 Mar 2023.

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [Hierarchical Needs Based Self-Adaptive Framework For Cooperative Multi-Robot System](https://mlsanity.com/paper/2004.10920/) (2020)
- [Needs-driven Heterogeneous Multi-Robot Cooperation in Rescue Missions](https://mlsanity.com/paper/2009.00288/) (2020)
- [How Can Robots Trust Each Other For Better Cooperation? A Relative Needs Entropy Based Robot-Robot Trust Assessment Model](https://mlsanity.com/paper/2105.07443/) (2021)
- [Game-theoretic Utility Tree for Multi-Robot Cooperative Pursuit Strategy](https://mlsanity.com/paper/2206.01109/) (2022)
- [A Game-Theoretic Utility Network for Cooperative Multi-Agent Decisions in Adversarial Environments](https://mlsanity.com/paper/2004.10950/) (2020)

## Built on by (most cited first, 1 of 1)

- [Edge Computing based Human-Robot Cognitive Fusion: A Medical Case Study in the Autism Spectrum Disorder Therapy](https://mlsanity.com/paper/2401.00776/) (2024)

## Abstract (arXiv)

Adopting reasonable strategies is challenging but crucial for an intelligent agent with limited resources working in hazardous, unstructured, and dynamic environments to improve the system's utility, decrease the overall cost, and increase mission success probability. This paper proposes a novel hierarchical strategy decomposition approach based on the Bayesian chain rule to separate an intricate policy into several simple sub-policies and organize their relationships as Bayesian strategy networks (BSN). We integrate this approach into the state-of-the-art DRL method -- soft actor-critic (SAC) and build the corresponding Bayesian soft actor-critic (BSAC) model by organizing several sub-policies as a joint policy. We compare the proposed BSAC method with the SAC and other state-of-the-art approaches such as TD3, DDPG, and PPO on the standard continuous control benchmarks -- Hopper-v2, Walker2d-v2, and Humanoid-v2 -- in MuJoCo with the OpenAI Gym environment. The results demonstrate that the promising potential of the BSAC method significantly improves training efficiency.