mlsanityData to 14 Oct 2026

Soft Actor-Critic Algorithms and Applications

arXiv 1812.05905 ↗First posted 13 Dec 2018cs.LG cs.AI cs.ROcited by 1,240

Not on the map. Why?

Cited by 424 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Model-free deep reinforcement learning (RL) algorithms have been successfully applied to a range of challenging sequential decision making and control tasks. However, these methods typically suffer from two major challenges: high sample complexity and brittleness to hyperparameters. Both of these challenges limit the applicability of such methods to real-world domains. In this paper, we describe Soft Actor-Critic (SAC), our recently introduced off-policy actor-critic algorithm based on the maximum entropy RL framework. In this framework, the actor aims to simultaneously maximize expected return and entropy. That is, to succeed at the task while acting as randomly as possible. We extend SAC to incorporate a number of modifications that accelerate training and improve stability with respect to the hyperparameters, including a constrained formulation that automatically tunes the temperature hyperparameter. We systematically evaluate SAC on a range of benchmark tasks, as well as real-world challenging tasks such as locomotion for a quadrupedal robot and robotic manipulation with a dexterous hand. With these improvements, SAC achieves state-of-the-art performance, outperforming prior on-policy and off-policy methods in sample-efficiency and asymptotic performance. Furthermore, we demonstrate that, in contrast to other off-policy algorithms, our approach is very stable, achieving similar performance across different random seeds. These results suggest that SAC is a promising candidate for learning in real-world robotics tasks.

Key facts. Cited by 1,240 papers in the map (data through 14 Oct 2026). First posted on arXiv 13 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 5; 2020: 9; 2021: 186; 2022: 204; 2023: 217; 2024: 241; 2025: 212; 2026: 166.

1,240 papers.

Sort:

By area:

  1. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  2. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  3. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  4. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  5. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  6. 2023Efficient Online Reinforcement Learning with Offline Datacited by 363
  7. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 327
  8. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 288
  9. 2022The Primacy Bias in Deep Reinforcement Learningcited by 247
  10. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 238
  11. 2023Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuningcited by 237
  12. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 215
38 more
  1. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  2. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 197
  3. 2022Online Decision Transformercited by 196
  4. 2019Imitation Learning via Off-Policy Distribution Matchingcited by 183
  5. 2021Reinforcement Learning with Prototypical Representationscited by 179
  6. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 176
  7. 2022Hybrid RL: Using Both Offline and Online Data Can Make RL Efficientcited by 161
  8. 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Modelcited by 151
  9. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 134
  10. 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMscited by 127
  11. 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 122
  12. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 122
  13. 2020Primal Wasserstein Imitation Learningcited by 121
  14. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 121
  15. 2022Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learningcited by 119
  16. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 116
  17. 2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 114
  18. 2021Recent Advances in Reinforcement Learning in Financecited by 110
  19. 2024Diffusion-based Reinforcement Learning via Q-weighted Variational Policy Optimizationcited by 110
  20. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous controlcited by 108
  21. 2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 107
  22. 2022Discovered Policy Optimisationcited by 104
  23. 2021Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithmscited by 101
  24. 2022Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimizationcited by 97
  25. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 95
  26. 2022Instruction-driven history-aware policies for robotic manipulationscited by 94
  27. 2019BAIL: Best-Action Imitation Learning for Batch Deep Reinforcement Learningcited by 91
  28. 2021Representation Matters: Offline Pretraining for Sequential Decision Makingcited by 91
  29. 2022q-Learning in Continuous Timecited by 91
  30. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
  31. 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
  32. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
  33. 2020COG: Connecting New Skills to Past Experience with Offline Reinforcement Learningcited by 83
  34. 2022Tutorial on amortized optimizationcited by 83
  35. 2023DeepACO: Neural-enhanced Ant Systems for Combinatorial Optimizationcited by 78
  36. 2021Demonstration-Guided Reinforcement Learning with Learned Skillscited by 76
  37. 2022Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matterscited by 76
  38. 2021Residual Pathway Priors for Soft Equivariance Constraintscited by 75

50 most cited of 1,240 All 1,240 (JSON)