mlsanityData to 7 Oct 2026

Automata-Conditioned Cooperative Multi-Agent Reinforcement Learning

arXiv 2511.02304 ↗First posted 4 Nov 2025cs.MA cs.AI cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

We study learning multi-task, multi-agent policies for cooperative, temporal objectives, under centralized training, decentralized execution. In this setting, using automata to represent tasks assigned to agents enables breaking down a team-level objective into simpler, smaller sub-tasks. However, existing approaches remain sample-inefficient and are limited to the single-task case, requiring retraining policies for each new task. In this work, we present Automata-Conditioned Cooperative Multi-Agent Reinforcement Learning (ACC-MARL), a framework for learning task-conditioned, decentralized team policies. We identify challenges to the feasibility of ACC-MARL, propose solutions, and prove that our approach is optimal. We further show that learned value functions can be used to assign tasks optimally at test time. Experiments demonstrate emergent task-aware, multi-step coordination among agents, such as pressing a button to unlock a door, holding the door, and short-circuiting tasks.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 24 of its 52 references that are in the map. First posted on arXiv 4 Nov 2025.

Builds on

24 of 52 references.

References · 24

  1. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 281
  2. 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 183
  3. 2017Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observabilitycited by 143
  4. 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
  5. 2022Discovered Policy Optimisationcited by 103
  6. 2021LTL2Action: Generalizing LTL Instructions for Multi-Task RLcited by 72
  7. 2020A Composable Specification Language for Reinforcement Learning Taskscited by 68
  8. 2021Compositional Reinforcement Learning from Logical Specificationscited by 65
  9. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 59
  10. 2021UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformerscited by 39
  11. 2020Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learningcited by 38
  12. 2020Reward Machines for Cooperative Multi-Agent Reinforcement Learningcited by 35
12 more
  1. 2021Multi-Agent Reinforcement Learning with Temporal Logic Specificationscited by 27
  2. 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 20
  3. 2024Compositional Automata Embeddings for Goal-Conditioned Reinforcement Learningcited by 18
  4. 2022ALMA: Hierarchical Learning for Composite Multi-Agent Taskscited by 17
  5. 2024LTL-Constrained Policy Optimization with Cycle Experience Replaycited by 10
  6. 2025One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learningcited by 8
  7. 2025Provably Correct Automata Embeddings for Optimal Automata-Conditioned Reinforcement Learningcited by 4
  8. 2024Multi-agent Path Finding for Timed Tasks using Evolutionary Gamescited by 3
  9. 2025Learning Symbolic Task Decompositions for Multi-Agent Teamscited by 3
  10. 2024An Evaluation of Massively Parallel Algorithms for DFA Minimizationcited by 1
  11. 2025CORD: Generalizable Cooperation via Role Diversitycited by 1
  12. 2025Multi-Task Multi-Agent Reinforcement Learning via Skill Graphscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learningcited by 1