# Learning Fair Policies in Decentralized Cooperative Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2012.09421 · HTML: https://mlsanity.com/paper/2012.09421/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2020.

## Built on by (most cited first, 30 of 40)

- [Two-sided fairness in rankings via Lorenz dominance](https://mlsanity.com/paper/2110.15781/) (2021)
- [Virtual Agent Economies](https://mlsanity.com/paper/2509.10147/) (2025)
- [What-is and How-to for Fairness in Machine Learning: A Survey, Reflection, and Perspective](https://mlsanity.com/paper/2206.04101/) (2022)
- [Survey on Fair Reinforcement Learning: Theory and Practice](https://mlsanity.com/paper/2205.10032/) (2022)
- [Optimizing generalized Gini indices for fairness in rankings](https://mlsanity.com/paper/2204.06521/) (2022)
- [Socially Fair Reinforcement Learning](https://mlsanity.com/paper/2208.12584/) (2022)
- [Fair Off-Policy Learning from Observational Data](https://mlsanity.com/paper/2303.08516/) (2023)
- [Recommendation Fairness: From Static to Dynamic](https://mlsanity.com/paper/2109.03150/) (2021)
- [Fairness in Preference-based Reinforcement Learning](https://mlsanity.com/paper/2306.09995/) (2023)
- [Fairness in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2405.06909/) (2024)
- [Principal-Agent Reinforcement Learning: Orchestrating AI Agents with Contracts](https://mlsanity.com/paper/2407.18074/) (2024)
- [Cooperative Multi-Agent Fairness and Equivariant Policies](https://mlsanity.com/paper/2106.05727/) (2021)
- [Achieving Fairness in Multi-Agent Markov Decision Processes Using Reinforcement Learning](https://mlsanity.com/paper/2306.00324/) (2023)
- [Tier Balancing: Towards Dynamic Fairness over Underlying Causal Factors](https://mlsanity.com/paper/2301.08987/) (2023)
- [Toward Structure Fairness in Dynamic Graph Embedding: A Trend-aware Dual Debiasing Approach](https://mlsanity.com/paper/2406.13201/) (2024)
- [Interactional Fairness in LLM Multi-Agent Systems: An Evaluation Framework](https://mlsanity.com/paper/2505.12001/) (2025)
- [Dealing with Non-Stationarity in MARL via Trust-Region Decomposition](https://mlsanity.com/paper/2102.10616/) (2021)
- [Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges](https://mlsanity.com/paper/2305.10091/) (2023)
- [Mediated Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2306.08419/) (2023)
- [MSRL: Distributed Reinforcement Learning with Dataflow Fragments](https://mlsanity.com/paper/2210.00882/) (2022)
- [Scalable Multi-Objective Reinforcement Learning with Fairness Guarantees using Lorenz Dominance](https://mlsanity.com/paper/2411.18195/) (2024)
- [DECAF: Learning to be Fair in Multi-agent Resource Allocation](https://mlsanity.com/paper/2502.04281/) (2025)
- [Learning Distributed and Fair Policies for Network Load Balancing as Markov Potential Game](https://mlsanity.com/paper/2206.01451/) (2022)
- [SCALES: From Fairness Principles to Constrained Decision-Making](https://mlsanity.com/paper/2209.10860/) (2022)
- [Fairness Aware Reinforcement Learning via Proximal Policy Optimization](https://mlsanity.com/paper/2502.03953/) (2025)
- [Beyond Predictions: A Participatory Framework for Multi-Stakeholder Decision-Making](https://mlsanity.com/paper/2502.08542/) (2025)
- [HypRL: Reinforcement Learning of Control Policies for Hyperproperties](https://mlsanity.com/paper/2504.04675/) (2025)
- [Learning Fair Pareto-Optimal Policies in Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2606.18111/) (2026)
- [Learning Rewards to Optimize Global Performance Metrics in Deep Reinforcement Learning](https://mlsanity.com/paper/2303.09027/) (2023)
- [Responsible Emergent Multi-Agent Behavior](https://mlsanity.com/paper/2311.01609/) (2023)

## Abstract (arXiv)

We consider the problem of learning fair policies in (deep) cooperative multi-agent reinforcement learning (MARL). We formalize it in a principled way as the problem of optimizing a welfare function that explicitly encodes two important aspects of fairness: efficiency and equity. As a solution method, we propose a novel neural network architecture, which is composed of two sub-networks specifically designed for taking into account the two aspects of fairness. In experiments, we demonstrate the importance of the two sub-networks for fair optimization. Our overall approach is general as it can accommodate any (sub)differentiable welfare function. Therefore, it is compatible with various notions of fairness that have been proposed in the literature (e.g., lexicographic maximin, generalized Gini social welfare function, proportional fairness). Our solution method is generic and can be implemented in various MARL settings: centralized training and decentralized execution, or fully decentralized. Finally, we experimentally validate our approach in various domains and show that it can perform much better than previous methods.