mlsanityData to 7 Oct 2026

Multi-Agent Reinforcement Learning via Agent-Specific Preference

arXiv 2608.08604 ↗First posted 9 Aug 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Multi-agent reinforcement learning (MARL) is a powerful framework for solving complex collaborative tasks, but it relies heavily on well-defined global reward functions. Designing such rewards is challenging, especially in systems with heterogeneous agents, where a single scalar objective may fail to capture diverse behaviors. In this paper, we introduce Multi-AGent Preference-Integrated lEarning (MAGPIE), which addresses these challenges through agent-specific preference modeling. Each agent is evaluated by a dedicated expert through preference signals, eliminating the need for global evaluation. We theoretically prove that optimizing these decentralized preferences converges to a Nash equilibrium policy. To integrate local preferences into a coherent global objective, we construct agent-specific reward models from preference data and combine them via a monotonic aggregation mechanism. We further prove that optimizing this aggregate reward model is equivalent to training the Nash equilibrium policy. Extensive experiments on benchmark multi-agent tasks and a sequential production line task show that MAGPIE achieves performance comparable to reward-engineered baselines, demonstrating its potential to facilitate policy learning in scenarios where precise reward engineering is impractical.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 58 references that are in the map. First posted on arXiv 9 Aug 2026.

Builds on

29 of 58 references.

References · 29

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  4. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  5. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  6. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  7. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  8. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  9. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
  10. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 257
  11. 2022SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learningcited by 96
  12. 2022Reinforcement Learning with Sparse Rewards using Guidance from Offline Demonstrationcited by 54
17 more
  1. 2019Reducing Overestimation Bias in Multi-Agent Domains Using Double Centralized Criticscited by 46
  2. 2021Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Reviewcited by 34
  3. 2025Multi-Agent Reinforcement Learning for Resources Allocation Optimization: A Surveycited by 14
  4. 2025Preference-based Multi-Objective Reinforcement Learningcited by 7
  5. 2022E-MAPP: Efficient Multi-Agent Reinforcement Learning with Parallel Program Guidancecited by 6
  6. 2025O-MAPL: Offline Multi-agent Preference Learningcited by 5
  7. 2025Human Implicit Preference-Based Policy Fine-tuning for Multi-Agent Reinforcement Learning in USV Swarmcited by 5
  8. 2024S-EPOA: Overcoming the Indistinguishability of Segments with Skill-Driven Preference-Based Reinforcement Learningcited by 4
  9. 2025CLARIFY: Contrastive Preference Reinforcement Learning for Untangling Ambiguous Queriescited by 3
  10. 2025STAIR: Addressing Stage Misalignment through Temporal-Aligned Preference Reinforcement Learningcited by 3
  11. 2026GlobeDiff: State Diffusion Process for Partial Observability in Multi-Agent Systemscited by 3
  12. 2021Decentralized Multi-Agent Reinforcement Learning: An Off-Policy Methodcited by 2
  13. 2025Beyond Shallow Behavior: Task-Efficient Value-Based Multi-Task Offline MARL via Skill Discoverycited by 1
  14. 2025Quality assessment of 3D human animation: Subjective and objective evaluationcited by 1
  15. 2025SENIOR: Efficient Query Selection and Preference-Guided Exploration in Preference-based Reinforcement Learningcited by 1
  16. 2025From Solo to Symphony: Orchestrating Multi-Agent Collaboration with Single-Agent Demoscited by 1
  17. 2026COLLIE: Guiding Skill Discovery in Semantically Coherent Latent Spacecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Preference-Guided Learning for Sparse-Reward Multi-Agent Reinforcement Learning
  2. 2026Improving LLM Collaboration via Multi-Agent Preference Learning
  3. 2025PB2: Preference Space Exploration via Population-Based Methods in Preference-Based Reinforcement Learning
  4. 2018Scalable agent alignment via reward modeling: a research direction
  5. 2025LLM Collaboration With Multi-Agent Reinforcement Learning
  6. 2025Multi-Agent Collaborative Reward Design for Enhancing Reasoning in Reinforcement Learning
  7. 2026From Correctness to Preference: A Framework for Personalized Agentic Reinforcement Learning
  8. 2026LEMUR: Learning to Align with Multi-Objective Reinforcement Learning from Preference Feedback