mlsanityData to 14 Oct 2026

Broad Critic Deep Actor Reinforcement Learning for Continuous Control

arXiv 2411.15806 ↗First posted 24 Nov 2024cs.LG cs.AIcited by 1

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

In the domain of continuous control, deep reinforcement learning (DRL) demonstrates promising results. However, the dependence of DRL on deep neural networks (DNNs) results in the demand for extensive data and increased computational cost. To address this issue, a novel hybrid actor-critic reinforcement learning (RL) framework is introduced. The proposed framework integrates the broad learning system (BLS) with DNN, aiming to merge the strengths of both distinct architectural paradigms. Specifically, the critic network employs BLS for rapid value estimation via ridge regression, while the actor network retains the DNN structure to optimize policy gradients. This hybrid design is generalizable and can enhance existing actor-critic algorithms. To demonstrate its versatility, the proposed framework is integrated into three widely used actor-critic algorithms -- deep deterministic policy gradient (DDPG), soft actor-critic (SAC), and twin delayed DDPG (TD3), resulting in BLS-augmented variants. Experimental results reveal that all BLS-enhanced versions surpass their original counterparts in terms of training efficiency and accuracy. These improvements highlight the suitability of the proposed framework for real-time control scenarios, where computational efficiency and rapid adaptation are critical.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 10 of its 51 references that are in the map. First posted on arXiv 24 Nov 2024.

Builds on

10 of 51 references.

References · 10

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  5. 2016OpenAI Gymcited by 1,736
  6. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 371
  7. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 151
  8. 2021Ensemble Bootstrapping for Q-Learningcited by 31
  9. 2022Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous Drivingcited by 24
  10. 2019QuaRL: Quantization for Fast and Environmentally Sustainable Reinforcement Learningcited by 19

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025SAC-NeRF: Adaptive Ray Sampling for Neural Radiance Fields via Soft Actor-Critic Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Model Free Deep Deterministic Policy Gradient Controller for Setpoint Tracking of Non-minimum Phase Systems
  2. 2025Expert-Free Online Transfer Learning in Multi-Agent Reinforcement Learning
  3. 2024Exploiting Estimation Bias in Clipped Double Q-Learning for Continous Control Reinforcement Learning Tasks
  4. 2024Value Improved Actor Critic Algorithms
  5. 2024Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning
  6. 2025Application of linear regression and quasi-Newton methods to the deep reinforcement learning in continuous action cases
  7. 2025Deterministic Policy Gradient for Reinforcement Learning with Continuous Time and State
  8. 2026Pseudo-Quantized Actor-Critic Algorithm for Robustness to Noisy Temporal Difference Error