Double Actor-Critic with TD Error-Driven Regularization in Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
To obtain better value estimation in reinforcement learning, we propose a novel algorithm based on the double actor-critic framework with temporal difference error-driven regularization, abbreviated as TDDR. TDDR employs double actors, with each actor paired with a critic, thereby fully leveraging the advantages of double critics. Additionally, TDDR introduces an innovative critic regularization architecture. Compared to classical deterministic policy gradient-based algorithms that lack a double actor-critic structure, TDDR provides superior estimation. Moreover, unlike existing algorithms with double actor-critic frameworks, TDDR does not introduce any additional hyperparameters, significantly simplifying the design and implementation process. Experiments demonstrate that TDDR exhibits strong competitiveness compared to benchmark algorithms in challenging continuous control tasks.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 27 of its 52 references that are in the map. First posted on arXiv 28 Sep 2024.
Builds on
27 of 52 references.
References · 27
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2015Trust Region Policy Optimizationcited by 2,503
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
- 2016OpenAI Gymcited by 1,684
- 2017Deep Reinforcement Learning that Matterscited by 727
- 2016Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement Learningcited by 106
- 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 73
- 2018How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experimentscited by 56
- 2020Softmax Deep Double Deterministic Policy Gradientscited by 31
- 2023Distributional Soft Actor-Critic with Three Refinementscited by 22
15 more
- 2020Diversity Actor-Critic: Sample-Aware Entropy Regularization for Sample-Efficient Explorationcited by 20
- 2021Efficient Continuous Control with Double Actors and Regularized Criticscited by 20
- 2018Deterministic Implementations for Reproducibility in Deep Reinforcement Learningcited by 19
- 2021Learning Pessimism for Robust and Efficient Off-Policy Reinforcement Learningcited by 18
- 2019Deep Conservative Policy Iterationcited by 14
- 2019Asynchronous Episodic Deep Deterministic Policy Gradient: Towards Continuous Control in Computationally Complex Environmentscited by 13
- 2021Model-based Chance-Constrained Reinforcement Learning via Separated Proportional-Integral Lagrangiancited by 13
- 2021Improved Soft Actor-Critic: Mixing Prioritized Off-Policy Samples with On-Policy Experiencecited by 8
- 2022Action Candidate Driven Clipped Double Q-learning for Discrete and Continuous Action Taskscited by 8
- 2024SPQR: Controlling Q-ensemble Independence with Spiked Random Model for Reinforcement Learningcited by 8
- 2021Value Activation for Bias Alleviation: Generalized-activated Deep Double Deterministic Policy Gradientscited by 6
- 2023Keep Various Trajectories: Promoting Exploration of Ensemble Policies in Continuous Controlcited by 5
- 2021ACERAC: Efficient reinforcement learning in fine time discretizationcited by 4
- 2021On the Estimation Bias in Double Q-Learningcited by 4
- 2022Mitigating Off-Policy Bias in Actor-Critic Methods with One-Step Q-learning: A Novel Correction Approachcited by 3
Built on by
Citing papers per year: 2025: 2.
2 papers.