mlsanityData to 7 Oct 2026

Multi-Agent Reinforcement Learning with Temporal Logic Specifications

arXiv 2102.00582 ↗First posted 1 Feb 2021cs.AI cs.LG cs.LOcited by 27

Not on the map. Why?

Abstract (arXiv)

In this paper, we study the problem of learning to satisfy temporal logic specifications with a group of agents in an unknown environment, which may exhibit probabilistic behaviour. From a learning perspective these specifications provide a rich formal language with which to capture tasks or objectives, while from a logic and automated verification perspective the introduction of learning capabilities allows for practical applications in large, stochastic, unknown environments. The existing work in this area is, however, limited. Of the frameworks that consider full linear temporal logic or have correctness guarantees, all methods thus far consider only the case of a single temporal logic specification and a single agent. In order to overcome this limitation, we develop the first multi-agent reinforcement learning technique for temporal logic specifications, which is also novel in its ability to handle multiple specifications. We provide correctness and convergence guarantees for our main algorithm - ALMANAC (Automaton/Logic Multi-Agent Natural Actor-Critic) - even when using function approximation. Alongside our theoretical results, we further demonstrate the applicability of our technique via a set of preliminary experiments.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 23 of its 64 references that are in the map. First posted on arXiv 1 Feb 2021.

Builds on

23 of 64 references.

References · 23

  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  2. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  3. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  4. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 189
  5. 2019On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shiftcited by 174
  6. 2016Reinforcement Learning With Temporal Logic Rewardscited by 114
  7. 2018Omega-Regular Objectives in Model-Free Reinforcement Learningcited by 74
  8. 2019Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learningcited by 74
  9. 2020A Composable Specification Language for Reinforcement Learning Taskscited by 68
  10. 2017Environment-Independent Task Specifications via GLTLcited by 67
  11. 2014Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraintscited by 53
  12. 2014A Learning Based Approach to Control Synthesis of Markov Decision Processes for Linear Temporal Logic Specificationscited by 49
11 more
  1. 2019Is the Policy Gradient a Gradient?cited by 40
  2. 2014Verification of Markov Decision Processes using Learning Algorithmscited by 38
  3. 2019Certified Reinforcement Learning with Logic Guidancecited by 35
  4. 2015Decentralized Q-Learning for Stochastic Teams and Gamescited by 34
  5. 2019PAC Statistical Model Checking for Markov Decision Processes and Stochastic Gamescited by 30
  6. 2018Logically-Constrained Neural Fitted Q-Iterationcited by 17
  7. 2020Reinforcement Learning of Control Policy for Linear Temporal Logic Specifications Using Limit-Deterministic Generalized B"uchi Automatacited by 11
  8. 2020Extended Markov Games to Learn Multiple Tasks in Multi-Agent Reinforcement Learningcited by 9
  9. 2018Equilibria-based Probabilistic Model Checking for Concurrent Stochastic Gamescited by 8
  10. 2018Automata Guided Reinforcement Learning With Demonstrationscited by 5
  11. 2020Reward Shaping for Reinforcement Learning with Omega-Regular Objectivescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 4; 2023: 8; 2024: 3; 2025: 3; 2026: 5.

27 papers.

Sort:

By area:

  1. 2021On the Expressivity of Markov Rewardcited by 71
  2. 2021Compositional Reinforcement Learning from Logical Specificationscited by 65
  3. 2021In a Nutshell, the Human Asked for This: Latent Goals for Following Temporal Specificationscited by 17
  4. 2024Three Dogmas of Reinforcement Learningcited by 14
  5. 2023LgTS: Dynamic Task Sampling using LLM-generated sub-goals for Reinforcement Learning Agentscited by 11
  6. 2022Specification-Guided Learning of Nash Equilibria with High Social Welfarecited by 8
  7. 2023Multi-Agent Reinforcement Learning Guided by Signal Temporal Logic Specificationscited by 6
  8. 2021Model-Free Learning of Safe yet Effective Controllerscited by 5
  9. 2023Reinforcement Learning with Knowledge Representation and Reasoning: A Brief Surveycited by 5
  10. 2023(Ir)rationality in AI: State of the Art, Research Challenges and Open Questionscited by 5
  11. 2023Standpoint Linear Temporal Logiccited by 4
  12. 2023Assume-Guarantee Reinforcement Learningcited by 4
15 more
  1. 2022DistSPECTRL: Distributing Specifications in Multi-Agent Reinforcement Learning Systemscited by 3
  2. 2024Logical Specifications-guided Dynamic Task Sampling for Reinforcement Learning Agentscited by 3
  3. 2022Distributed Control using Reinforcement Learning with Temporal-Logic-Based Reward Shapingcited by 2
  4. 2023Optimal Control of Logically Constrained Partially Observable and Multi-Agent Markov Decision Processescited by 2
  5. 2024Tractable Offline Learning of Regular Decision Processescited by 2
  6. 2025HypRL: Reinforcement Learning of Control Policies for Hyperpropertiescited by 2
  7. 2026PlatoLTL: Scaling LTL-Guided Multi-Task RLcited by 2
  8. 2023Multi-Agent Verification and Control with Probabilistic Model Checkingcited by 1
  9. 2025Automata-Conditioned Cooperative Multi-Agent Reinforcement Learningcited by 1
  10. 2026SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learningcited by 1
  11. 2022CatlNet: Learning Communication and Coordination Policies from CaTL+ Specificationscited by 0
  12. 2025Adversarial Reinforcement Learning for Offensive and Defensive Agents in a Simulated Zero-Sum Network Environmentcited by 0
  13. 2026HyPOLE: Hyperproperty-Guided Multi-Agent Reinforcement Learning under Partial Observationcited by 0
  14. 2026The syntax and semantics of goalscited by 0
  15. 2026Epistemic-Probabilistic Model for Guarded Multi-Agent LLM Coordinationcited by 0