mlsanityData to 14 Oct 2026

G2MAF: Test-Time Gradient Guidance for Multi-Agent Flow Policies

arXiv 2609.31286 ↗First posted 25 Sep 2026cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Offline multi-agent RL

Abstract (arXiv)

Offline multi-agent reinforcement learning (MARL) learns cooperative policies from fixed datasets without further environment interaction and a learned policy is frozen at deployment. Such a frozen policy typically proposes a single joint action and executes it directly at deployment time. However, this one-shot deployment often commits to a suboptimal proposal, even when better nearby alternatives remain consistent with the behavior data. To address this issue, we propose Gradient Guided Multi Agent Flow (G2MAF), a refinement framework for optimizing joint policies at test-time. G2MAF applies one globally normalized, projected critic gradient to guide and coordinate all agents' corrections while keeping the action both feasible and close to the frozen policy proposal. Across 24 MPE and SMAC settings, its canonical variant improves 20 frozen settings, with mean relative gains of 9.2% on MPE and 8.9% on SMAC, with model inference latency increased by about 6% only.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Offline multi-agent RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 26 of its 35 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

26 of 35 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026CoFlow: Coordinated Few-Step Flow for Offline Multi-Agent Decision Making2 May
  2. 2025OM2P: Offline Multi-Agent Mean-Flow Policy8 Aug
  3. 2024Offline Multi-Agent Reinforcement Learning via In-Sample Sequential Policy Optimization10 Dec
  4. 2024Exploiting Structure in Offline Multi-Agent RL: The Benefits of Low Interaction Rank1 Oct
  5. 2024Putting Data at the Centre of Offline Multi-Agent Reinforcement Learning18 Sep
  6. 2023MADiff: Offline Multi-agent Learning with Diffusion Models27 May
  7. 2021Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification22 Nov
  8. 2021Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learning7 Jun

Other references · 18

  1. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  2. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  3. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  4. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
  5. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  6. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 516
  7. 2025Flow Q-Learningcited by 184
  8. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 182
  9. 2025Energy-Weighted Flow Matching for Offline Reinforcement Learningcited by 73
  10. 2025Diffusion Guidance Is a Controllable Policy Improvement Operatorcited by 58
  11. 2026Q-learning with Adjoint Matchingcited by 37
  12. 2025One-Step Flow Policy Mirror Descentcited by 21
6 more
  1. 2025FlowQ: Energy-Guided Flow Policies for Offline Reinforcement Learningcited by 15
  2. 2025Multi-agent Coordination via Flow Matchingcited by 10
  3. 2025Guided Flow Policy: Learning from High-Value Actions in Offline Reinforcement Learningcited by 9
  4. 2025Test-time Offline Reinforcement Learning on Goal-related Experiencecited by 8
  5. 2026Reinforcement Learning via Value Gradient Flowcited by 5
  6. 2026Value-Guidance MeanFlow for Offline Multi-Agent Reinforcement Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026MA-FPPO: Multi-Agent Flow-Pretrained Policy Optimization
  2. 2026MA-WAM: Multi-Agent World-Action Model for Test-Time Planning
  3. 2026Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning
  4. 2026Flowing Faster to Coordinate: One-Step Online Multi-Agent Flow Policies
  5. 2026Test-time Multi-agent Coordination by Decomposed Value Gradient Flow
  6. 2026Descent-Guided Policy Gradient for Scalable Cooperative Multi-Agent Learning
  7. 2025Offline Multi-agent Reinforcement Learning via Sequential Score Decomposition
  8. 2024ComaDICE: Offline Cooperative Multi-Agent Reinforcement Learning with Stationary Distribution Shift Regularization