mlsanityData to 7 Oct 2026

Soft-Robust Actor-Critic Policy-Gradient

arXiv 1803.04848 ↗First posted 11 Mar 2018cs.LG cs.AI stat.MLcited by 37

Not on the map. Why?

Abstract (arXiv)

Robust Reinforcement Learning aims to derive optimal behavior that accounts for model uncertainty in dynamical systems. However, previous studies have shown that by considering the worst case scenario, robust policies can be overly conservative. Our soft-robust framework is an attempt to overcome this issue. In this paper, we present a novel Soft-Robust Actor-Critic algorithm (SR-AC). It learns an optimal policy with respect to a distribution over an uncertainty set and stays robust to model uncertainty but avoids the conservativeness of robust strategies. We show the convergence of SR-AC and test the efficiency of our approach on different domains by comparing it against regular learning methods and their robust formulations.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 7; 2022: 7; 2023: 11; 2024: 5; 2025: 3; 2026: 3.

37 papers.

Sort:

By area:

  1. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
  2. 2022Robust Reinforcement Learning using Offline Datacited by 95
  3. 2021Sample Complexity of Robust Reinforcement Learning with a Generative Modelcited by 82
  4. 2021Twice regularized MDPs and the equivalence between robustness and regularizationcited by 46
  5. 2023Improved Sample Complexity Bounds for Distributionally Robust Reinforcement Learningcited by 44
  6. 2021Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challengescited by 40
  7. 2023Natural Actor-Critic for Robust Reinforcement Learning with Function Approximationcited by 35
  8. 2023Policy Gradient for Rectangular Robust Markov Decision Processescited by 29
  9. 2020Bayesian Robust Optimization for Imitation Learningcited by 28
  10. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  11. 2024Distributionally Robust Off-Dynamics Reinforcement Learning: Provable Efficiency with Linear Function Approximationcited by 25
  12. 2024Sample-Efficient Robust Multi-Agent Reinforcement Learning in the Face of Environmental Uncertaintycited by 24
25 more
  1. 2022Model-Based Offline Reinforcement Learning with Pessimism-Modulated Dynamics Beliefcited by 22
  2. 2023Distributionally Robust Model-based Reinforcement Learning with Large State Spacescited by 17
  3. 2022Robust Policy Learning over Multiple Uncertainty Setscited by 16
  4. 2023Soft Robust MDPs and Risk-Sensitive MDPs: Equivalence, Policy Gradient, and Sample Complexitycited by 16
  5. 2023Single-Trajectory Distributionally Robust Reinforcement Learningcited by 15
  6. 2023Risk-Averse Model Uncertainty for Distributionally Robust Safe Reinforcement Learningcited by 12
  7. 2021Policy Gradient Bayesian Robust Optimization for Imitation Learningcited by 11
  8. 2023Optimal Transport Perturbations for Safe Reinforcement Learning with Robustness Guaranteescited by 9
  9. 2022Robust Reinforcement Learning in Continuous Control Tasks with Uncertainty Set Regularizationcited by 7
  10. 2023Value-Distributional Model-Based Reinforcement Learningcited by 5
  11. 2022RASR: Risk-Averse Soft-Robust MDPs with EVaR and Entropic Riskcited by 4
  12. 2021Lyapunov Robust Constrained-MDPs: Soft-Constrained Robustly Stable Policy Optimization under Model Uncertaintycited by 3
  13. 2025DR-SAC: Distributionally Robust Soft Actor-Critic for Reinforcement Learning under Uncertaintycited by 3
  14. 2024Solving Multi-Model MDPs by Coordinate Ascent and Dynamic Programmingcited by 2
  15. 2021Evaluating the progress of Deep Reinforcement Learning in the real world: aligning domain-agnostic and domain-specific researchcited by 1
  16. 2023Twice Regularized Markov Decision Processes: The Equivalence between Robustness and Regularizationcited by 1
  17. 2025Empirical Study on Robustness and Resilience in Cooperative Multi-Agent Reinforcement Learningcited by 1
  18. 2025Long-Horizon Model-Based Offline Reinforcement Learning Without Explicit Conservatismcited by 1
  19. 2026Uncertainty-Aware Reward Discounting for Mitigating Reward Hackingcited by 1
  20. 2022User-Oriented Robust Reinforcement Learningcited by 0
  21. 2023On Practical Robust Reinforcement Learning: Practical Uncertainty Set and Double-Agent Algorithmcited by 0
  22. 2024Percentile Criterion Optimization in Offline Reinforcement Learningcited by 0
  23. 2024DyPNIPP: Predicting Environment Dynamics for RL-based Robust Informative Path Planningcited by 0
  24. 2026Distributionally Robust Cooperative Multi-Agent Reinforcement Learning via Robust Value Factorizationcited by 0
  25. 2026Counterfactually Safe Reinforcement Learningcited by 0