mlsanityData to 14 Oct 2026

HypeMARL: Multi-Agent Reinforcement Learning For High-Dimensional, Parametric, and Distributed Systems

arXiv 2509.16709 ↗First posted 20 Sep 2025cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning has recently emerged as a promising feedback control strategy for complex dynamical systems governed by partial differential equations (PDEs). When dealing with distributed, high-dimensional problems in state and control variables, multi-agent reinforcement learning (MARL) has been proposed as a scalable approach for breaking the curse of dimensionality. In particular, through decentralized training and execution, multiple agents cooperate to steer the system towards a target configuration, relying solely on local state and reward information. However, the principle of locality may become a limiting factor whenever a collective, nonlocal behavior of the agents is crucial to maximize the reward function, as typically happens in PDE-constrained optimal control problems. In this work, we propose HypeMARL: a decentralized MARL algorithm tailored to the control of high-dimensional, parametric, and distributed systems. HypeMARL employs hypernetworks to effectively parametrize the agents' policies and value functions with respect to the system parameters and the agents' relative positions, encoded by sinusoidal positional encoding. Through the application on challenging control problems, such as density and flow control, we show that HypeMARL (i) can effectively control systems through a collective behavior of the agents, outperforming state-of-the-art decentralized MARL, (ii) can efficiently deal with parametric dependencies, (iii) requires minimal hyperparameter tuning and (iv) can reduce the amount of expensive environment interactions by a factor of ~10 thanks to its model-based extension, MB-HypeMARL, which relies on computationally efficient deep learning-based surrogate models approximating the dynamics locally, with minimal deterioration of the policy performance.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 29 of its 56 references that are in the map. First posted on arXiv 20 Sep 2025.

Builds on

29 of 56 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  6. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  7. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  8. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  9. 2017Convolutional Sequence to Sequence Learningcited by 552
  10. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  11. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  12. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
17 more
  1. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
  2. 2017Deep Reinforcement Learning: An Overviewcited by 333
  3. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 294
  4. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 261
  5. 2018Mean Field Multi-Agent Reinforcement Learningcited by 207
  6. 2018An Introduction to Deep Reinforcement Learningcited by 206
  7. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 186
  8. 2015Multiagent Cooperation and Competition with Deep Reinforcement Learningcited by 184
  9. 2023A Brief Review of Hypernetworks in Deep Learningcited by 102
  10. 2021Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharingcited by 87
  11. 2018Deep Reinforcement Learning for Swarm Systemscited by 45
  12. 2020Continual Model-Based Reinforcement Learning with Hypernetworkscited by 44
  13. 2022Hypernetworks in Meta-Reinforcement Learningcited by 42
  14. 2021Recomposing the Reinforcement Learning Building Blocks with Hypernetworkscited by 35
  15. 2023Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you needcited by 27
  16. 2023Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learningcited by 24
  17. 2024Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policiescited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026HypEMBER: Hypernetwork-based Ensemble for Robust Policy Learning of Parametrized Dynamical Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024HyperMARL: Adaptive Hypernetworks for Multi-Agent RL
  2. 2025HypeRL: Hypernetwork-Based Reinforcement Learning for Control of Parametrized Dynamical Systems
  3. 2024MARL-LNS: Cooperative Multi-agent Reinforcement Learning via Large Neighborhoods Search
  4. 2025Optimization without Future Compromises? Decentralized Coordination via Collective and Reinforcement Learning
  5. 2025Multi-Agent Reinforcement Learning for Sample-Efficient Deep Neural Network Mapping
  6. 2025Structured Cooperative Multi-Agent Reinforcement Learning: a Bayesian Network Perspective
  7. 2024ComaDICE: Offline Cooperative Multi-Agent Reinforcement Learning with Stationary Distribution Shift Regularization
  8. 2025Reinforcement Networks: novel framework for collaborative Multi-Agent Reinforcement Learning tasks