mlsanityData to 7 Oct 2026

Generalising Discrete Action Spaces with Conditional Action Trees

arXiv 2104.07294 ↗First posted 15 Apr 2021cs.LG cs.AI stat.MLcited by 6

Not on the map. Why?

Abstract (arXiv)

There are relatively few conventions followed in reinforcement learning (RL) environments to structure the action spaces. As a consequence the application of RL algorithms to tasks with large action spaces with multiple components require additional effort to adjust to different formats. In this paper we introduce em Conditional Action Trees with two main objectives: (1) as a method of structuring action spaces in RL to generalise across several action space specifications, and (2) to formalise a process to significantly reduce the action space by decomposing it into multiple sub-spaces, favoring a multi-staged decision making approach. We show several proof-of-concept experiments validating our scheme, ranging from environments with basic discrete action spaces to those with large combinatorial action spaces commonly found in RTS-style games.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 15 of its 23 references that are in the map. First posted on arXiv 15 Apr 2021.

Builds on

15 of 23 references.

References · 15

  1. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  2. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  3. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  4. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  5. 2019The StarCraft Multi-Agent Challengecited by 589
  6. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
  7. 2020A Closer Look at Invalid Action Masking in Policy Gradient Algorithmscited by 160
  8. 2020Graph Constrained Reinforcement Learning for Natural Language Action Spacescited by 80
  9. 2019Combinatorial Optimization by Graph Pointer Networks and Hierarchical Reinforcement Learningcited by 78
  10. 2015Reinforcement Learning with Parameterized Actionscited by 61
  11. 2019Hybrid Actor-Critic Reinforcement Learning in Parameterized Action Spacecited by 53
  12. 2020Action Space Shaping in Deep Reinforcement Learningcited by 45
3 more
  1. 2020Griddly: A platform for AI research in gamescited by 31
  2. 2019Autoregressive Policies for Continuous Control Deep Reinforcement Learningcited by 13
  3. 2019Comparing Observation and Action Representations for Deep Reinforcement Learning in μRTScited by 2

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 2; 2025: 2.

6 papers.

  1. 2024Entity-based Reinforcement Learning for Autonomous Cyber Defencecited by 11
  2. 2022GriddlyJS: A Web IDE for Reinforcement Learningcited by 4
  3. 2025Optimizing Navigation And Chemical Application in Precision Agriculture With Deep Reinforcement Learning And Conditional Action Treecited by 2
  4. 2023PyTAG: Challenges and Opportunities for Reinforcement Learning in Tabletop Gamescited by 1
  5. 2024PyTAG: Tabletop Games for Multi-Agent Reinforcement Learningcited by 1
  6. 2025Who Gets the Reward & Who Gets the Blame? Evaluation-Aligned Training Signals for Multi-LLM Agentscited by 0