mlsanityData to 7 Oct 2026

Distributed Distributional Deterministic Policy Gradients

arXiv 1804.08617 ↗First posted 23 Apr 2018cs.LG cs.AI stat.MLcited by 195

Not on the map. Why?

Cited by 32 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This work adopts the very successful distributional perspective on reinforcement learning and adapts it to the continuous control setting. We combine this within a distributed framework for off-policy learning in order to develop what we call the Distributed Distributional Deep Deterministic Policy Gradient algorithm, D4PG. We also combine this technique with a number of additional, simple improvements such as the use of N-step returns and prioritized experience replay. Experimentally we examine the contribution of each of these individual components, and show how they interact, as well as their combined contributions. Our results show that across a wide variety of simple control tasks, difficult manipulation tasks, and a set of hard obstacle-based locomotion tasks the D4PG algorithm achieves state of the art performance.

Key facts. Cited by 195 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 3; 2020: 5; 2021: 50; 2022: 36; 2023: 35; 2024: 31; 2025: 15; 2026: 19.

195 papers.

Sort:

By area:

  1. 2023Mastering Diverse Domains through World Modelscited by 1,193
  2. 2022A Generalist Agentcited by 802
  3. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  4. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 347
  5. 2020Primal Wasserstein Imitation Learningcited by 120
  6. 2020Hyperparameter Selection for Offline Reinforcement Learningcited by 113
  7. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
  8. 2021Online and Offline Reinforcement Learning by Planning with a Learned Modelcited by 90
  9. 2021AndroidEnv: A Reinforcement Learning Platform for Androidcited by 90
  10. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 81
  11. 2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 79
  12. 2021Conservative Offline Distributional Reinforcement Learningcited by 78
38 more
  1. 2021Learning and Planning in Complex Action Spacescited by 76
  2. 2020Offline Learning from Demonstrations and Unlabeled Experiencecited by 72
  3. 2021What Matters for Adversarial Imitation Learning?cited by 70
  4. 2021Risk-Averse Offline Reinforcement Learningcited by 65
  5. 2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 63
  6. 2020RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learningcited by 57
  7. 2024Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agentscited by 56
  8. 2023Learning to Generate Better Than Your LLMcited by 49
  9. 2024Learning Multimodal Behaviors from Scratch with Diffusion Policy Gradientcited by 49
  10. 2019Task-Relevant Adversarial Imitation Learningcited by 45
  11. 2024Diffusion Policies creating a Trust Region for Offline Reinforcement Learningcited by 41
  12. 2019Positive-Unlabeled Reward Learningcited by 36
  13. 2023Boosting Continuous Control with Consistency Policycited by 36
  14. 2021Regularized Behavior Value Estimationcited by 32
  15. 2021Autoregressive Dynamics Models for Offline Policy Evaluation and Optimizationcited by 32
  16. 2024LMAct: A Benchmark for In-Context Imitation Learning with Long Multimodal Demonstrationscited by 32
  17. 2023An Analysis of Quantile Temporal-Difference Learningcited by 30
  18. 2023Distributional GFlowNets with Quantile Flowscited by 28
  19. 2021Residual Reinforcement Learning from Demonstrationscited by 27
  20. 2021Active Offline Policy Selectioncited by 26
  21. 2024Quantization-Aware Imitation-Learning for Resource-Efficient Robotic Controlcited by 25
  22. 2022Continuous MDP Homomorphisms and Homomorphic Policy Gradientcited by 24
  23. 2021Task-Agnostic Morphology Evolutioncited by 22
  24. 2021Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racingcited by 22
  25. 2021On Instrumental Variable Regression for Deep Offline Policy Evaluationcited by 22
  26. 2021Bayesian Sequential Optimal Experimental Design for Nonlinear Models Using Policy Gradient Reinforcement Learningcited by 22
  27. 2023A Survey on Physics Informed Reinforcement Learning: Review and Open Problemscited by 22
  28. 2023Distributional Soft Actor-Critic with Three Refinementscited by 22
  29. 2022Solving Continuous Control via Q-learningcited by 21
  30. 2021Efficient Continuous Control with Double Actors and Regularized Criticscited by 20
  31. 2023MAC-PO: Multi-Agent Experience Replay via Collective Priority Optimizationcited by 20
  32. 2024ManiSkill-HAB: A Benchmark for Low-Level Manipulation in Home Rearrangement Taskscited by 20
  33. 2021Revisiting Peng's Q(λ) for Modern Reinforcement Learningcited by 19
  34. 2022Asynchronous Reinforcement Learning for Real-Time Control of Physical Robotscited by 19
  35. 2023ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for Last-Iterate Convergence in Constrained MDPscited by 19
  36. 2026SOP: A Scalable Online Post-Training System for Vision-Language-Action Modelscited by 19
  37. 2019Scaling data-driven robotics with reward sketching and batch reinforcement learningcited by 18
  38. 2025One-Step Flow Policy Mirror Descentcited by 18

50 most cited of 195 All 195 (JSON)