# DSAC: Distributional Soft Actor-Critic for Risk-Sensitive Reinforcement Learning

arXiv: https://arxiv.org/abs/2004.14547 · HTML: https://mlsanity.com/paper/2004.14547/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Apr 2020.

## Built on by (most cited first, 8 of 8)

- [Value Activation for Bias Alleviation: Generalized-activated Deep Double Deterministic Policy Gradients](https://mlsanity.com/paper/2112.11216/) (2021)
- [Optimistic Exploration for Risk-Averse Constrained Reinforcement Learning](https://mlsanity.com/paper/2507.08793/) (2025)
- [Towards an Autonomous Test Driver: High-Performance Driver Modeling via Reinforcement Learning](https://mlsanity.com/paper/2412.03803/) (2024)
- [Uncertainty-Aware Safety-Critical Decision and Control for Autonomous Vehicles at Unsignalized Intersections](https://mlsanity.com/paper/2505.19939/) (2025)
- [Risk-Sensitive and Robust Model-Based Reinforcement Learning and Planning](https://mlsanity.com/paper/2304.00573/) (2023)
- [Risk-Aware Reinforcement Learning for Mobile Manipulation](https://mlsanity.com/paper/2603.04579/) (2026)
- [Distributional Soft Bellman Operator under the Cram'er Geometry](https://mlsanity.com/paper/2607.17897/) (2026)
- [A Risk-Sensitive and Uncertainty-Aware Decision-Making and Control Framework for Safe and Robust Autonomous Driving](https://mlsanity.com/paper/2609.09650/) (2026)

## Abstract (arXiv)

We present Distributional Soft Actor-Critic (DSAC), a distributional reinforcement learning (RL) algorithm that combines the strengths of distributional information of accumulated rewards and entropy-driven exploration from Soft Actor-Critic (SAC) algorithm. DSAC models the randomness in both action and rewards, surpassing baseline performances on various continuous control tasks. Unlike standard approaches that solely maximize expected rewards, we propose a unified framework for risk-sensitive learning, one that optimizes the risk-related objective while balancing entropy to encourage exploration. Extensive experiments demonstrate DSAC's effectiveness in enhancing agent performances for both risk-neutral and risk-sensitive control tasks.