mlsanityData to 14 Oct 2026

Comparative Analysis of Parameterized Action Actor-Critic Reinforcement Learning Algorithms for Web Search Match Plan Generation

arXiv 2510.03064 ↗First posted 3 Oct 2025cs.LG cs.AIcited by 1

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

This study evaluates the performance of Soft Actor Critic (SAC), Greedy Actor Critic (GAC), and Truncated Quantile Critics (TQC) in high-dimensional decision-making tasks using fully observable environments. The focus is on parametrized action (PA) spaces, eliminating the need for recurrent networks, with benchmarks Platform-v0 and Goal-v0 testing discrete actions linked to continuous action-parameter spaces. Hyperparameter optimization was performed with Microsoft NNI, ensuring reproducibility by modifying the codebase for GAC and TQC. Results show that Parameterized Action Greedy Actor-Critic (PAGAC) outperformed other algorithms, achieving the fastest training times and highest returns across benchmarks, completing 5,000 episodes in 41:24 for the Platform game and 24:04 for the Robot Soccer Goal game. Its speed and stability provide clear advantages in complex action spaces. Compared to PASAC and PATQC, PAGAC demonstrated superior efficiency and reliability, making it ideal for tasks requiring rapid convergence and robust performance. Future work could explore hybrid strategies combining entropy-regularization with truncation-based methods to enhance stability and expand investigations into generalizability.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 9 of its 19 references that are in the map. First posted on arXiv 3 Oct 2025.

Builds on

9 of 19 references.

References · 9

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2019A Theoretical Analysis of Deep Q-Learningcited by 215
  6. 2020Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Criticscited by 165
  7. 2019Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spacescited by 24
  8. 2018Greedy Actor-Critic: A New Conditional Cross-Entropy Method for Policy Improvementcited by 12
  9. 2018Optimizing Query Evaluations using Reinforcement Learning for Web Searchcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Comparative Study of Multi-Agent Actor-Critic Algorithms in Parameterized Action Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Closing the Gap: Achieving Global Convergence (Last Iterate) of Actor-Critic under Markovian Sampling with Neural Network Parametrization
  2. 2025Parallel Heuristic Search as Inference for Actor-Critic Reinforcement Learning Models
  3. 2026Efficient Soft Actor-Critic with LLM-Based Action-Level Guidance for Continuous Control
  4. 2025Chunking the Critic: A Transformer-based Soft Actor-Critic with N-Step Returns
  5. 2025Effective Reinforcement Learning Control using Conservative Soft Actor-Critic
  6. 2026SMAC: Score-Matched Actor-Critics for Robust Offline-to-Online Transfer
  7. 2025Gradual Transition from Bellman Optimality Operator to Bellman Operator in Online Reinforcement Learning
  8. 2024Generalizing soft actor-critic algorithms to discrete action spaces