mlsanityData to 7 Oct 2026

Improving Exploration in Soft-Actor-Critic with Normalizing Flows Policies

arXiv 1906.02771 ↗First posted 6 Jun 2019cs.LG cs.AI stat.MLcited by 37

Not on the map. Why?

Abstract (arXiv)

Deep Reinforcement Learning (DRL) algorithms for continuous action spaces are known to be brittle toward hyperparameters as well as cutbeingsample inefficient. Soft Actor Critic (SAC) proposes an off-policy deep actor critic algorithm within the maximum entropy RL framework which offers greater stability and empirical gains. The choice of policy distribution, a factored Gaussian, is motivated by cutchosen dueits easy re-parametrization rather than its modeling power. We introduce Normalizing Flow policies within the SAC framework that learn more expressive classes of policies than simple factored Gaussians. cutWe also present a series of stabilization tricks that enable effective training of these policies in the RL setting.We show empirically on continuous grid world tasks that our approach increases stability and is better suited to difficult exploration in sparse reward settings.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 11; 2023: 5; 2024: 9; 2025: 2; 2026: 4.

37 papers.

Sort:

By area:

  1. 2023Guided Flows for Generative Modeling and Decision Makingcited by 99
  2. 2021Greedification Operators for Policy Optimization: Investigating Forward and Reverse KL Divergencescited by 40
  3. 2024Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robotscited by 32
  4. 2022Universal approximation property of invertible neural networkscited by 30
  5. 2024Maximum Entropy Reinforcement Learning via Energy-Based Normalizing Flowcited by 25
  6. 2022Action Noise in Off-Policy Deep Reinforcement Learning: Impact on Exploration and Performancecited by 21
  7. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  8. 2022Flow-based Recurrent Belief State Learning for POMDPscited by 16
  9. 2024FlowPG: Action-constrained Policy Gradient with Normalizing Flowscited by 16
  10. 2021Learning a subspace of policies for online adaptation in Reinforcement Learningcited by 14
  11. 2025Generative Models in Decision Making: A Surveycited by 14
  12. 2022Latent State Marginalization as a Low-cost Approach for Improving Explorationcited by 13
25 more
  1. 2022Revisiting Discrete Soft Actor-Criticcited by 11
  2. 2023Generative Modelling of Stochastic Actions with Arbitrary Constraints in Reinforcement Learningcited by 10
  3. 2021Implicitly Regularized RL with Implicit Q-Valuescited by 9
  4. 2024Monotone, Bi-Lipschitz, and Polyak-Lojasiewicz Networkscited by 9
  5. 2021Improved Soft Actor-Critic: Mixing Prioritized Off-Policy Samples with On-Policy Experiencecited by 8
  6. 2022TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speedcited by 8
  7. 2022Trust Region Policy Optimization with Optimal Transport Discrepancies: Duality and Algorithm for Continuous Actionscited by 8
  8. 2021Robust normalizing flows using Bernstein-type polynomialscited by 6
  9. 2022Reward Bonuses with Gain Scheduling Inspired by Iterative Deepening Searchcited by 5
  10. 2022CEIP: Combining Explicit and Implicit Priors for Reinforcement Learning with Demonstrationscited by 4
  11. 2024How to Choose a Reinforcement-Learning Algorithmcited by 3
  12. 2022Fail-Safe Adversarial Generative Imitation Learningcited by 2
  13. 2023Soft Actor-Critic Algorithm with Truly-satisfied Inequality Constraintcited by 2
  14. 2025Leveraging Constraint Violation Signals For Action-Constrained Reinforcement Learningcited by 2
  15. 2023On Learning the Tail Quantiles of Driving Behavior Distributions via Quantile Regression and Flowscited by 1
  16. 2024Towards Interpretable Reinforcement Learning with Constrained Normalizing Flow Policiescited by 1
  17. 2024Design of Restricted Normalizing Flow towards Arbitrary Stochastic Policy with Computational Efficiencycited by 1
  18. 2026Warp RL: Reshaping Base Policy Distributions for Dynamics Adaptationcited by 1
  19. 2026NFTR: From Provable Mode-Averaging to Geodesic Subgoal Selection in Offline Goal-Conditioned RLcited by 1
  20. 2021Out-of-Distribution Detection of Melanoma using Normalizing Flowscited by 0
  21. 2022Exploration with Multi-Sample Target Values for Distributional Reinforcement Learningcited by 0
  22. 2023LLQL: Logistic Likelihood Q-Learning for Reinforcement Learningcited by 0
  23. 2024Soft Actor-Critic with Beta Policy via Implicit Reparameterization Gradientscited by 0
  24. 2026APC-RL: Exceeding Data-Driven Behavior Priors with Adaptive Policy Compositioncited by 0
  25. 2026QHyer: Q-conditioned Hybrid Attention-mamba Transformer for Offline Goal-conditioned RLcited by 0