mlsanityData to 7 Oct 2026

Reinforcement Learning in Factored Action Spaces using Tensor Decompositions

arXiv 2110.14538 ↗First posted 27 Oct 2021cs.LG cs.MAcited by 6

Not on the map. Why?

Abstract (arXiv)

We present an extended abstract for the previously published work TESSERACT [Mahajan et al., 2021], which proposes a novel solution for Reinforcement Learning (RL) in large, factored action spaces using tensor decompositions. The goal of this abstract is twofold: (1) To garner greater interest amongst the tensor research community for creating methods and analysis for approximate RL, (2) To elucidate the generalised setting of factored action spaces where tensor decompositions can be used. We use cooperative multi-agent reinforcement learning scenario as the exemplary setting where the action space is naturally factored across agents and learning becomes intractable without resorting to approximation on the underlying hypothesis space for candidate solutions.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 24 of its 48 references that are in the map. First posted on arXiv 27 Oct 2021.

Builds on

24 of 48 references.

References · 24

  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  2. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  3. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  4. 2019The StarCraft Multi-Agent Challengecited by 589
  5. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
  6. 2017A Survey of Model Compression and Acceleration for Deep Neural Networkscited by 298
  7. 2016Tensor Decomposition for Signal Processing and Machine Learningcited by 223
  8. 2019MAVEN: Multi-Agent Variational Explorationcited by 181
  9. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  10. 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
  11. 2015On the Expressive Power of Deep Learning: A Tensor Analysiscited by 97
  12. 2020Qatten: A General Framework for Cooperative Multiagent Reinforcement Learningcited by 81
12 more
  1. 2017Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectivescited by 53
  2. 2021Tensor Methods in Computer Vision and Deep Learningcited by 52
  3. 2019Factorized Higher-Order CNNs with an Application to Spatio-Temporal Emotion Estimationcited by 40
  4. 2020UneVEn: Universal Value Exploration for Multi-Agent Reinforcement Learningcited by 32
  5. 2021Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learningcited by 25
  6. 2019T-Net: Parametrizing Fully Convolutional Nets with a Single High-Order Tensorcited by 22
  7. 2018Factorized Q-Learning for Large-Scale Multi-Agent Systemscited by 19
  8. 2013The Sample-Complexity of General Reinforcement Learningcited by 18
  9. 2019Incremental multi-domain learning with network latent tensor factorizationcited by 16
  10. 2020Spectral Learning on Matrices and Tensorscited by 16
  11. 2017Symmetry Learning for Function Approximation in Reinforcement Learningcited by 14
  12. 2016Reinforcement Learning in Rich-Observation MDPs using Spectral Methodscited by 13

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 2.

6 papers.

  1. 2023Structure in Deep Reinforcement Learning: A Survey and Open Problemscited by 16
  2. 2024Efficient Monte Carlo Tree Search via On-the-Fly State-Conditioned Action Abstractioncited by 6
  3. 2024Stochastic Q-learning for Large Discrete Action Spacescited by 5
  4. 2023Dynamic Neighborhood Construction for Structured Large Discrete Action Spacescited by 4
  5. 2022MAN: Multi-Action Networks Learningcited by 2
  6. 2023Diverse Policy Optimization for Structured Action Spacecited by 1