mlsanityData to 7 Oct 2026

Growing Action Spaces

arXiv 1906.12266 ↗First posted 28 Jun 2019cs.LG cs.AI stat.MLcited by 29

Not on the map. Why?

Abstract (arXiv)

In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress. In this work, we use a curriculum of progressively growing action spaces to accelerate learning. We assume the environment is out of our control, but that the agent may set an internal curriculum by initially restricting its action space. Our approach uses off-policy reinforcement learning to estimate optimal value functions for multiple action spaces simultaneously and efficiently transfers data, value estimates, and state representations from restricted action spaces to the full task. We show the efficacy of our approach in proof-of-concept control tasks and on challenging large-scale StarCraft micromanagement tasks with large, multi-agent action spaces.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 7; 2023: 3; 2024: 6; 2025: 5; 2026: 2.

29 papers.

Sort:

By area:

  1. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  2. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  3. 2021Reinforcement Learning for Adaptive Mesh Refinementcited by 37
  4. 2021Is Bang-Bang Control All You Need? Solving Continuous Control with Bernoulli Policiescited by 37
  5. 2022DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systemscited by 27
  6. 2022Solving Continuous Control via Q-learningcited by 21
  7. 2021Deep hierarchical reinforcement agents for automated penetration testingcited by 19
  8. 2023Prompt Engineering Through the Lens of Optimal Controlcited by 16
  9. 2021Solving Large-Scale Extensive-Form Network Security Games via Neural Fictitious Self-Playcited by 7
  10. 2022Bingham Policy Parameterization for 3D Rotations in Reinforcement Learningcited by 7
  11. 2022Behaviour-Diverse Automatic Penetration Testing: A Curiosity-Driven Multi-Objective Deep Reinforcement Learning Approachcited by 7
  12. 2024Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolutioncited by 6
17 more
  1. 2024Stochastic Q-learning for Large Discrete Action Spacescited by 5
  2. 2024BraVE: Offline Reinforcement Learning for Discrete Combinatorial Action Spacescited by 3
  3. 2024An Investigation of Offline Reinforcement Learning in Factorisable Action Spacescited by 3
  4. 2023Cooperative Multi-Agent Reinforcement Learning for Inventory Managementcited by 2
  5. 2025Online Decision Making with Generative Action Setscited by 2
  6. 2022Forecasting Evolution of Clusters in Game Agents with Hebbian Learningcited by 1
  7. 2023Diverse Policy Optimization for Structured Action Spacecited by 1
  8. 2025Action Space Reduction Strategies for Reinforcement Learning in Autonomous Drivingcited by 1
  9. 2025Rethinking the Role of Dynamic Sparse Training for Scalable Deep Reinforcement Learningcited by 1
  10. 2026Vocabulary Dropout for Curriculum Diversity in LLM Co-Evolutioncited by 1
  11. 2021More Efficient Exploration with Symbolic Priors on Action Sequence Equivalencescited by 0
  12. 2022GAIL-PT: A Generic Intelligent Penetration Testing Framework with Generative Adversarial Imitation Learningcited by 0
  13. 2024Constructing Ancestral Recombination Graphs through Reinforcement Learningcited by 0
  14. 2024Learning in complex action spaces without policy gradientscited by 0
  15. 2025SAINT: Attention-Based Policies for Discrete Combinatorial Action Spacescited by 0
  16. 2025Where to Intervene: Action Selection in Deep Reinforcement Learningcited by 0
  17. 2026Improving and Accelerating Offline RL in Large Discrete Action Spaces with Structured Policy Initializationcited by 0