# Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Survey

arXiv: https://arxiv.org/abs/1909.02964 · HTML: https://mlsanity.com/paper/1909.02964/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2019.

## Built on by (most cited first, 30 of 47)

- [A Practical Guide to Multi-Objective Reinforcement Learning and Planning](https://mlsanity.com/paper/2103.09568/) (2021)
- [Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards](https://mlsanity.com/paper/2306.04488/) (2023)
- [Scalar reward is not enough: A response to Silver, Singh, Precup and Sutton (2021)](https://mlsanity.com/paper/2112.15422/) (2021)
- [Virtual Agent Economies](https://mlsanity.com/paper/2509.10147/) (2025)
- [Causal Multi-Agent Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2111.06721/) (2021)
- [Autonomy and Intelligence in the Computing Continuum: Challenges, Enablers, and Future Directions for Orchestration](https://mlsanity.com/paper/2205.01423/) (2022)
- [Expected Scalarised Returns Dominance: A New Solution Concept for Multi-Objective Decision Making](https://mlsanity.com/paper/2106.01048/) (2021)
- [MOMAland: A Set of Benchmarks for Multi-Objective Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2407.16312/) (2024)
- [Risk Aware and Multi-Objective Decision Making with Distributional Monte Carlo Tree Search](https://mlsanity.com/paper/2102.00966/) (2021)
- [Monte Carlo Tree Search Algorithms for Risk-Aware and Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2211.13032/) (2022)
- [Navigating the Social Welfare Frontier: Portfolios for Multi-objective Reinforcement Learning](https://mlsanity.com/paper/2502.09724/) (2025)
- [Reward-Sharing Relational Networks in Multi-Agent Reinforcement Learning as a Framework for Emergent Behavior](https://mlsanity.com/paper/2207.05886/) (2022)
- [Human-in-the-Loop Policy Optimization for Preference-Based Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2401.02160/) (2024)
- [Utility-Based Reinforcement Learning: Unifying Single-objective and Multi-objective Reinforcement Learning](https://mlsanity.com/paper/2402.02665/) (2024)
- [Preference Communication in Multi-Objective Normal-Form Games](https://mlsanity.com/paper/2111.09191/) (2021)
- [Multi-Objective Coordination Graphs for the Expected Scalarised Returns with Generative Flow Models](https://mlsanity.com/paper/2207.00368/) (2022)
- [Utility Theory based Cognitive Modeling in the Application of Robotics: A Survey](https://mlsanity.com/paper/2306.09445/) (2023)
- [Regulation Games for Trustworthy Machine Learning](https://mlsanity.com/paper/2402.03540/) (2024)
- [A survey of air combat behavior modeling using machine learning](https://mlsanity.com/paper/2404.13954/) (2024)
- [Exploring the Impact of Tunable Agents in Sequential Social Dilemmas](https://mlsanity.com/paper/2101.11967/) (2021)
- [Multi-Objective reward generalization: Improving performance of Deep Reinforcement Learning for applications in single-asset trading](https://mlsanity.com/paper/2203.04579/) (2022)
- [Designing a Classifier for Active Fire Detection from Multispectral Satellite Imagery Using Neural Architecture Search](https://mlsanity.com/paper/2410.05425/) (2024)
- [Beyond Predictions: A Participatory Framework for Multi-Stakeholder Decision-Making](https://mlsanity.com/paper/2502.08542/) (2025)
- [Exploring Equity of Climate Policies using Multi-Agent Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2505.01115/) (2025)
- [All unconstrained strongly convex problems are weakly simplicial](https://mlsanity.com/paper/2106.12704/) (2021)
- [A Review of the Deep Sea Treasure problem as a Multi-Objective Reinforcement Learning Benchmark](https://mlsanity.com/paper/2110.06742/) (2021)
- [Improving Urban Mobility: using artificial intelligence and new technologies to connect supply and demand](https://mlsanity.com/paper/2204.03570/) (2022)
- [gTLO: A Generalized and Non-linear Multi-Objective Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2204.04988/) (2022)
- [Federated Learning in UAV-Enhanced Networks: Joint Coverage and Convergence Time Optimization](https://mlsanity.com/paper/2308.16889/) (2023)
- [Deep Multi-Objective Reinforcement Learning for Utility-Based Infrastructural Maintenance Optimization](https://mlsanity.com/paper/2406.06184/) (2024)

## Abstract (arXiv)

The majority of multi-agent system (MAS) implementations aim to optimise agents' policies with respect to a single objective, despite the fact that many real-world problem domains are inherently multi-objective in nature. Multi-objective multi-agent systems (MOMAS) explicitly consider the possible trade-offs between conflicting objective functions. We argue that, in MOMAS, such compromises should be analysed on the basis of the utility that these compromises have for the users of a system. As is standard in multi-objective optimisation, we model the user utility using utility functions that map value or return vectors to scalar values. This approach naturally leads to two different optimisation criteria: expected scalarised returns (ESR) and scalarised expected returns (SER). We develop a new taxonomy which classifies multi-objective multi-agent decision making settings, on the basis of the reward structures, and which and how utility functions are applied. This allows us to offer a structured view of the field, to clearly delineate the current state-of-the-art in multi-objective multi-agent decision making approaches and to identify promising directions for future research. Starting from the execution phase, in which the selected policies are applied and the utility for the users is attained, we analyse which solution concepts apply to the different settings in our taxonomy. Furthermore, we define and discuss these solution concepts under both ESR and SER optimisation criteria. We conclude with a summary of our main findings and a discussion of many promising future research directions in multi-objective multi-agent systems.