mlsanityData to 7 Oct 2026

Large Language Model Guided Incentive Aware Reward Design for Cooperative Multi-Agent Reinforcement Learning

arXiv 2603.24324 ↗First posted 25 Mar 2026cs.LG cs.AI cs.SYcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Designing effective auxiliary rewards for cooperative multi-agent systems remains challenging, as misaligned incentives can induce suboptimal coordination, particularly when sparse task rewards provide insufficient grounding for coordinated behavior. This study introduces an autonomous reward design framework that uses large language models (LLMs) to synthesize executable reward programs from environment instrumentation. The procedure constrains candidate programs within a formal validity envelope and trains policies from scratch using Multi-Agent Proximal Policy Optimization (MAPPO) under a fixed computational budget. The candidates are then evaluated on the basis of their performance, and selection across generations solely based on the sparse task returns. The framework is evaluated in four Overcooked-AI layouts characterized by varying levels of corridor congestion, handoff dependencies, and structural asymmetries. The proposed reward design approach consistently yields higher task returns and delivery counts, with the most pronounced gains observed in environments dominated by interaction bottlenecks. Diagnostic analysis of the synthesized shaping components reveals stronger interdependence in action selection and improved signal alignment in coordination-intensive tasks. These results demonstrate that the proposed LLM-guided reward search framework mitigates the need for manual engineering while producing shaping signals compatible with cooperative learning under finite budgets.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 18 of its 29 references that are in the map. First posted on arXiv 25 Mar 2026.

Builds on

18 of 29 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2017Deep reinforcement learning from human preferencescited by 3,846
  4. 2020Learning to summarize from human feedbackcited by 2,562
  5. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  6. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  7. 2023Large Language Models as Optimizerscited by 830
  8. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  9. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  10. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  11. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 345
  12. 2017Inverse Reward Designcited by 246
6 more
  1. 2020"Other-Play" for Zero-Shot Coordinationcited by 194
  2. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 153
  3. 2021Collaborating with Humans without Human Datacited by 139
  4. 2020Learning to Utilize Shaping Rewards: A New Approach of Reward Shapingcited by 112
  5. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 99
  6. 2023Behavior Alignment via Reward Function Optimizationcited by 16

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026The End of Reward Engineering: How LLMs Are Redefining Multi-Agent Coordination
  2. 2026Reward-Driven LLM Agent Workflows: Synthesizing POMDP Routing and Self-Correction for Autonomous Decision-Making
  3. 2025MAESTRO: Multi-Agent Environment Shaping through Task and Reward Optimization
  4. 2026LLM-ALSO: LLM-Driven Adaptive Learning-Signal Optimization for Multi-Agent Reinforcement Learning
  5. 2025Boosting Universal LLM Reward Design through Heuristic Reward Observation Space Evolution
  6. 2025Cooper: Co-Optimizing Policy and Reward Models in Reinforcement Learning for Large Language Models
  7. 2025Reinforcement Learning-Augmented LLM Agents for Collaborative Decision Making and Performance Optimization
  8. 2025An Automated Reinforcement Learning Reward Design Framework with Large Language Model for Cooperative Platoon Coordination