mlsanityData to 14 Oct 2026

Actor-Critic with Active Importance Sampling

arXiv 2605.07094 ↗First posted 8 May 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

This paper introduces the Active-Importance-Sampling Actor-Critic (AISAC) algorithm, an extension of the Actor-Critic framework for reducing variance in policy gradient estimation. AISAC optimizes the behavior policy to minimize gradient variance while preserving unbiased gradient estimates. Using importance sampling principles, the algorithm adapts the behavior policy toward efficient data collection distributions aligned with target policy gradients. For continuous action spaces, AISAC employs Gaussian behavior policies optimized through cross-entropy minimization. We provide theoretical analysis demonstrating variance reduction and unbiasedness. Experiments on Inverted Pendulum and Half Cheetah tasks show improved learning speed, sample efficiency, and training stability compared to standard Actor-Critic methods. Results indicate that optimizing the behavior policy improves both target policy updates and critic estimation accuracy across different hyperparameter settings. AISAC accelerates convergence and stabilizes reinforcement learning training, making it promising for real-world applications. Future work includes integration with advanced algorithms such as Soft Actor-Critic and TD3 for more complex environments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 5 of its 10 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

5 of 10 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Policy Gradient with Active Importance Sampling9 May

Other references · 4

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2017Data-Efficient Policy Evaluation Through Behavior Policy Searchcited by 26

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Importance Weighted Actor-Critic for Optimal Conservative Offline Reinforcement Learning
  2. 2026FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Control
  3. 2025Langevin Soft Actor-Critic: Efficient Exploration through Uncertainty-Driven Critic Learning
  4. 2026Bayesian policy gradient and actor-critic algorithms
  5. 2025Wasserstein Barycenter Soft Actor-Critic
  6. 2024Compatible Gradient Approximations for Actor-Critic Algorithms
  7. 2025Effective Reinforcement Learning Control using Conservative Soft Actor-Critic
  8. 2024Actor-Critic Reinforcement Learning with Phased Actor