mlsanityData to 14 Oct 2026

Action Branching Architectures for Deep Reinforcement Learning

arXiv 1711.08946 ↗First posted 24 Nov 2017cs.LG cs.AIcited by 92

Not on the map. Why?

Cited by 31 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Discrete-action algorithms have been central to numerous recent successes of deep reinforcement learning. However, applying these algorithms to high-dimensional action tasks requires tackling the combinatorial increase of the number of possible actions with the number of action dimensions. This problem is further exacerbated for continuous-action tasks that require fine control of actions via discretization. In this paper, we propose a novel neural architecture featuring a shared decision module followed by several network branches, one for each action dimension. This approach achieves a linear increase of the number of network outputs with the number of degrees of freedom by allowing a level of independence for each individual action dimension. To illustrate the approach, we present a novel agent, called Branching Dueling Q-Network (BDQ), as a branching variant of the Dueling Double Deep Q-Network (Dueling DDQN). We evaluate the performance of our agent on a set of challenging continuous control tasks. The empirical results show that the proposed agent scales gracefully to environments with increasing action dimensionality and indicate the significance of the shared decision module in coordination of the distributed action branches. Furthermore, we show that the proposed agent performs competitively against a state-of-the-art continuous control algorithm, Deep Deterministic Policy Gradient (DDPG).

Key facts. Cited by 92 papers in the map (data through 14 Oct 2026). First posted on arXiv 24 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 14; 2022: 6; 2023: 21; 2024: 20; 2025: 17; 2026: 14.

92 papers.

Sort:

By area:

  1. 2024Behavior Generation with Latent Actionscited by 160
  2. 2023Leveraging Factored Action Spaces for Efficient Offline Reinforcement Learning in Healthcarecited by 42
  3. 2025On Entropy Control in LLM-RL Algorithmscited by 40
  4. 2021Is Bang-Bang Control All You Need? Solving Continuous Control with Bernoulli Policiescited by 38
  5. 2023Action-Quantized Offline Reinforcement Learning for Robotic Skill Learningcited by 36
  6. 2021Learning Large Neighborhood Search Policy for Integer Programmingcited by 35
  7. 2022Solving Continuous Control via Q-learningcited by 24
  8. 2021Continuous Control with Action Quantization from Demonstrationscited by 22
  9. 2021A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directionscited by 21
  10. 2021Deep hierarchical reinforcement agents for automated penetration testingcited by 19
  11. 2021Learning Practically Feasible Policies for Online 3D Bin Packingcited by 18
  12. 2023AdaptSim: Task-Driven Simulation Adaptation for Sim-to-Real Transfercited by 18
38 more
  1. 2023B-Coder: Value-Based Deep Reinforcement Learning for Program Synthesiscited by 17
  2. 2024Continuous Control with Coarse-to-fine Reinforcement Learningcited by 17
  3. 2021DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunitiescited by 15
  4. 2021Centralizing State-Values in Dueling Networks for Multi-Robot Reinforcement Learning Mapless Navigationcited by 13
  5. 2022Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfacescited by 12
  6. 2024The Evolution of Reinforcement Learning in Quantitative Finance: A Surveycited by 11
  7. 2023SMARLA: A Safety Monitoring Approach for Deep Reinforcement Learning Agentscited by 10
  8. 2021Implicitly Regularized RL with Implicit Q-Valuescited by 9
  9. 2024Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolutioncited by 7
  10. 2023Large-Scale Traffic Signal Control Using Constrained Network Partition and Adaptive Deep Reinforcement Learningcited by 6
  11. 2023No Prior Mask: Eliminate Redundant Action for Deep Reinforcement Learningcited by 6
  12. 2024REValueD: Regularised Ensemble Value-Decomposition for Factorisable Markov Decision Processescited by 6
  13. 2024Predicting Long-Term Human Behaviors in Discrete Representations via Physics-Guided Diffusioncited by 6
  14. 2021Chanakya: Learning Runtime Decisions for Adaptive Real-Time Perceptioncited by 5
  15. 2023Inducing Stackelberg Equilibrium through Spatio-Temporal Sequential Decision-Making in Multi-Agent Reinforcement Learningcited by 5
  16. 2023Multi-UAV Speed Control with Collision Avoidance and Handover-aware Cell Association: DRL with Action Branchingcited by 5
  17. 2024Stochastic Q-learning for Large Discrete Action Spacescited by 5
  18. 2021Dynamic Multichannel Access via Multi-agent Reinforcement Learning: Throughput and Fairness Guaranteescited by 4
  19. 2023Exploring Deep Reinforcement Learning for Holistic Smart Building Controlcited by 4
  20. 2023Dynamic Neighborhood Construction for Structured Large Discrete Action Spacescited by 4
  21. 2024Discretizing Continuous Action Space with Unimodal Probability Distributions for On-Policy Reinforcement Learningcited by 4
  22. 2021Soft Actor-Critic With Integer Actionscited by 3
  23. 2023Optimization Theory Based Deep Reinforcement Learning for Resource Allocation in Ultra-Reliable Wireless Networked Control Systemscited by 3
  24. 2024BraVE: Offline Reinforcement Learning for Discrete Combinatorial Action Spacescited by 3
  25. 2024An Investigation of Offline Reinforcement Learning in Factorisable Action Spacescited by 3
  26. 2022PROMPT: Learning Dynamic Resource Allocation Policies for Network Applicationscited by 2
  27. 2022MAN: Multi-Action Networks Learningcited by 2
  28. 2023Cooperative Multi-Agent Reinforcement Learning for Inventory Managementcited by 2
  29. 2023FedPEAT: Convergence of Federated Learning, Parameter-Efficient Fine Tuning, and Emulator Assisted Tuning for Artificial Intelligence Foundation Models with Mobile Edge Computingcited by 2
  30. 2024Cooperative Multi-Agent Deep Reinforcement Learning in Content Ranking Optimizationcited by 2
  31. 2025Dynamic Optimizations of LLM Ensembles with Two-Stage Reinforcement Learning Agentscited by 2
  32. 2021An Approach to Partial Observability in Games: Learning to Both Act and Observecited by 1
  33. 2023Diverse Policy Optimization for Structured Action Spacecited by 1
  34. 2023Soft Decomposed Policy-Critic: Bridging the Gap for Effective Continuous Control with Discrete RLcited by 1
  35. 2023Deep Reinforcement Learning for the Heat Transfer Control of Pulsating Impinging Jetscited by 1
  36. 2024Deep Reinforcement Learning for Controlled Traversing of the Attractor Landscape of Boolean Models in the Context of Cellular Reprogrammingcited by 1
  37. 2024Simplex Decomposition for Portfolio Allocation Constraints in Reinforcement Learningcited by 1
  38. 2024Integrating Reinforcement Learning and Model Predictive Control with Applications to Microgridscited by 1

50 most cited of 92 All 92 (JSON)