mlsanityData to 7 Oct 2026

Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spaces

arXiv 1905.04388 ↗First posted 10 May 2019cs.LG stat.MLcited by 22

Not on the map. Why?

Abstract (arXiv)

Parameterised actions in reinforcement learning are composed of discrete actions with continuous action-parameters. This provides a framework for solving complex domains that require combining high-level actions with flexible control. The recent P-DQN algorithm extends deep Q-networks to learn over such action spaces. However, it treats all action-parameters as a single joint input to the Q-network, invalidating its theoretical foundations. We analyse the issues with this approach and propose a novel method, multi-pass deep Q-networks, or MP-DQN, to address them. We empirically demonstrate that MP-DQN significantly outperforms P-DQN and other previous algorithms in terms of data efficiency and converged policy performance on the Platform, Robot Soccer Goal, and Half Field Offense domains.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 2; 2024: 2; 2025: 7; 2026: 5.

22 papers.

Sort:

By area:

  1. 2021HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representationcited by 33
  2. 2022Meta-Learning Parameterized Skillscited by 7
  3. 2024Model-based Reinforcement Learning for Parameterized Action Spacescited by 7
  4. 2022Reinforcement learning for traffic signal control in hybrid action spacecited by 5
  5. 2023Diverse Policy Optimization for Structured Action Spacecited by 1
  6. 2024Detecting Continuous Integration Skip : A Reinforcement Learning-based Approachcited by 1
  7. 2025Comparative Analysis of Parameterized Action Actor-Critic Reinforcement Learning Algorithms for Web Search Match Plan Generationcited by 1
  8. 2025Learning Parameterized Skills from Demonstrationscited by 1
  9. 2022Deep Multi-Agent Reinforcement Learning with Hybrid Action Spaces based on Maximum Entropycited by 0
  10. 2022A further exploration of deep Multi-Agent Reinforcement Learning with Hybrid Action Spacecited by 0
  11. 2022Computational Discovery of Energy-Efficient Heat Treatment for Microstructure Design using Deep Reinforcement Learningcited by 0
  12. 2023Mixed-Integer Optimal Control via Reinforcement Learning: A Case Study on Hybrid Electric Vehicle Energy Managementcited by 0
10 more
  1. 2025Learn A Flexible Exploration Model for Parameterized Action Markov Decision Processescited by 0
  2. 2025A Parallel Hybrid Action Space Reinforcement Learning Model for Real-world Adaptive Traffic Signal Controlcited by 0
  3. 2025Multi-Agent Deep Reinforcement Learning for Safe Autonomous Driving with RICS-Assisted MECcited by 0
  4. 2025Strategic Communication under Threat: Learning Information Trade-offs in Pursuit-Evasion Gamescited by 0
  5. 2025Context-Sensitive Abstractions for Reinforcement Learning with Parameterized Actionscited by 0
  6. 2026Breaking the Grid: Distance-Guided Reinforcement Learning in Large Discrete Action Spacescited by 0
  7. 2026Learning Hybrid-Control Policies for High-Precision In-Contact Manipulation Under Uncertaintycited by 0
  8. 2026Knowledge- and Gradient-Guided Reinforcement Learning for Parametrized Action Markov Decision Processescited by 0
  9. 2026Comparative Study of Multi-Agent Actor-Critic Algorithms in Parameterized Action Reinforcement Learningcited by 0
  10. 2026Reinforcement Learning to Accelerate Primal-Dual Hybrid Gradient for Linear Programmingcited by 0