mlsanityData to 7 Oct 2026

Approximated Multi-Agent Fitted Q Iteration

arXiv 2104.09343 ↗First posted 19 Apr 2021cs.LG cs.SY eess.SYcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

We formulate an efficient approximation for multi-agent batch reinforcement learning, the approximated multi-agent fitted Q iteration (AMAFQI). We present a detailed derivation of our approach. We propose an iterative policy search and show that it yields a greedy policy with respect to multiple approximations of the centralized, learned Q-function. In each iteration and policy evaluation, AMAFQI requires a number of computations that scales linearly with the number of agents whereas the analogous number of computations increase exponentially for the fitted Q iteration (FQI), a commonly used approaches in batch reinforcement learning. This property of AMAFQI is fundamental for the design of a tractable multi-agent approach. We evaluate the performance of AMAFQI and compare it to FQI in numerical simulations. The simulations illustrate the significant computation time reduction when using AMAFQI instead of FQI in multi-agent problems and corroborate the similar performance of both approaches.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 6 of its 38 references that are in the map. First posted on arXiv 19 Apr 2021.

Builds on

6 of 38 references.

References · 6

  1. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  2. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  3. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 189
  4. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 173
  5. 2019A Review of Cooperative Multi-Agent Deep Reinforcement Learningcited by 147
  6. 2018Cooperative and Distributed Reinforcement Learning of Drones for Field Coveragecited by 20

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2020Q* Approximation Schemes for Batch Reinforcement Learning: A Theoretical Comparison
  2. 2024Fitted Q-Iteration via Max-Plus-Linear Approximation
  3. 2024Switching the Loss Reduces the Cost in Batch (Offline) Reinforcement Learning
  4. 2020Batch Value-function Approximation with Only Realizability
  5. 2022Offline Reinforcement Learning with Differentiable Function Approximation is Provably Efficient
  6. 2026Finite-Sample Theory for Fitted Q-Iteration When Actions Are Functions
  7. 2025Scalable Multi-Agent Offline Reinforcement Learning and the Role of Information
  8. 2023Sample-Efficiency in Multi-Batch Reinforcement Learning: The Need for Dimension-Dependent Adaptivity