mlsanityData to 14 Oct 2026

MOMA-AC: A preference-driven actor-critic framework for continuous multi-objective multi-agent reinforcement learning

arXiv 2511.18181 ↗First posted 22 Nov 2025cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper addresses a critical gap in Multi-Objective Multi-Agent Reinforcement Learning (MOMARL) by introducing the first dedicated inner-loop actor-critic framework for continuous state and action spaces: Multi-Objective Multi-Agent Actor-Critic (MOMA-AC). Building on single-objective, single-agent algorithms, we instantiate this framework with Twin Delayed Deep Deterministic Policy Gradient (TD3) and Deep Deterministic Policy Gradient (DDPG), yielding MOMA-TD3 and MOMA-DDPG. The framework combines a multi-headed actor network, a centralised critic, and an objective preference-conditioning architecture, enabling a single neural network to encode the Pareto front of optimal trade-off policies for all agents across conflicting objectives in a continuous MOMARL setting. We also outline a natural test suite for continuous MOMARL by combining a pre-existing multi-agent single-objective physics simulator with its multi-objective single-agent counterpart. Evaluating cooperative locomotion tasks in this suite, we show that our framework achieves statistically significant improvements in expected utility and hypervolume relative to outer-loop and independent training baselines, while demonstrating stable scalability as the number of agents increases. These results establish our framework as a foundational step towards robust, scalable multi-objective policy learning in continuous multi-agent domains.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 16 of its 39 references that are in the map. First posted on arXiv 22 Nov 2025.

Builds on

16 of 39 references.

References · 16

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  6. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  7. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  8. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
  9. 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 256
  10. 2020FACMAC: Factored Multi-Agent Centralised Policy Gradientscited by 178
  11. 2018Evolution-Guided Policy Gradient in Reinforcement Learningcited by 83
  12. 2024Reinforcement Learning Enhanced LLMs: A Surveycited by 71
4 more
  1. 2021Scalar reward is not enough: A response to Silver, Singh, Precup and Sutton (2021)cited by 55
  2. 2023Sample-Efficient Multi-Objective Learning via Generalized Policy Improvement Prioritizationcited by 48
  3. 2019Collaborative Evolutionary Reinforcement Learningcited by 37
  4. 2023Multi-Objective Reinforcement Learning Based on Decomposition: A Taxonomy and Frameworkcited by 31

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Sampling-Based Coordination-Informed Multi-Objective Multi-Robot Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MOMAland: A Set of Benchmarks for Multi-Objective Multi-Agent Reinforcement Learning
  2. 2026Learning Coordinated Preference for Multi-Objective Multi-Agent Reinforcement Learning
  3. 2025Multi-Actor Multi-Critic Deep Deterministic Reinforcement Learning with a Novel Q-Ensemble Method
  4. 2025Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approach
  5. 2026MO-MIX: Multi-Objective Multi-Agent Cooperative Decision-Making With Deep Reinforcement Learning
  6. 2025Scalable Multi-Objective Robot Reinforcement Learning through Gradient Conflict Resolution
  7. 2024Natural Policy Gradient and Actor Critic Methods for Constrained Multi-Task Reinforcement Learning
  8. 2026Comparative Study of Multi-Agent Actor-Critic Algorithms in Parameterized Action Reinforcement Learning