# Hypernetwork-based approach for optimal composition design in partially controlled multi-agent systems

arXiv: https://arxiv.org/abs/2502.12605 · HTML: https://mlsanity.com/paper/2502.12605/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 18 Feb 2025.

## Builds on (most cited first, 15 of 15)

- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.08839/) (2020)
- [Mean Field Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1802.05438/) (2018)
- [A Brief Review of Hypernetworks in Deep Learning](https://mlsanity.com/paper/2306.06955/) (2023)
- [Recomposing the Reinforcement Learning Building Blocks with Hypernetworks](https://mlsanity.com/paper/2106.06842/) (2021)
- [A Survey on Autonomous Vehicle Control in the Era of Mixed-Autonomy: From Physics-Based to AI-Guided Driving Policy Learning](https://mlsanity.com/paper/2007.05156/) (2020)
- [End-to-End Learning and Intervention in Games](https://mlsanity.com/paper/2010.13834/) (2020)
- [Adaptive Incentive Design with Multi-Agent Meta-Gradient Reinforcement Learning](https://mlsanity.com/paper/2112.10859/) (2021)
- [Multi-Agent Contracts](https://mlsanity.com/paper/2211.05434/) (2022)
- [Learning How to Dynamically Route Autonomous Vehicles on Shared Roads](https://mlsanity.com/paper/1909.03664/) (2019)
- [Reward Design for Driver Repositioning Using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2002.06723/) (2020)
- [Population-size-Aware Policy Optimization for Mean-Field Games](https://mlsanity.com/paper/2302.03364/) (2023)
- [IQ-Flow: Mechanism Design for Inducing Cooperative Behavior to Self-Interested Agents in Sequential Social Dilemmas](https://mlsanity.com/paper/2302.14604/) (2023)
- [Algorithmic Contract Design with Reinforcement Learning Agents](https://mlsanity.com/paper/2408.09686/) (2024)
- [Bayesian Optimization Framework for Efficient Fleet Design in Autonomous Multi-Robot Exploration](https://mlsanity.com/paper/2408.11751/) (2024)

## Abstract (arXiv)

Partially Controlled Multi-Agent Systems (PCMAS) are comprised of controllable agents, managed by a system designer, and uncontrollable agents, operating autonomously. This study addresses an optimal composition design problem in PCMAS, which involves the system designer's problem, determining the optimal number and policies of controllable agents, and the uncontrollable agents' problem, identifying their best-response policies. Solving this bi-level optimization problem is computationally intensive, as it requires repeatedly solving multi-agent reinforcement learning problems under various compositions for both types of agents. To address these challenges, we propose a novel hypernetwork-based framework that jointly optimizes the system's composition and agent policies. Unlike traditional methods that train separate policy networks for each composition, the proposed framework generates policies for both controllable and uncontrollable agents through a unified hypernetwork. This approach enables efficient information sharing across similar configurations, thereby reducing computational overhead. Additional improvements are achieved by incorporating reward parameter optimization and mean action networks. Using real-world New York City taxi data, we demonstrate that our framework outperforms existing methods in approximating equilibrium policies. Our experimental results show significant improvements in key performance metrics, such as order response rate and served demand, highlighting the practical utility of controlling agents and their potential to enhance decision-making in PCMAS.