# Multi-Agent Learning of Numerical Methods for Hyperbolic PDEs with Factored Dec-MDP

arXiv: https://arxiv.org/abs/2205.15716 · HTML: https://mlsanity.com/paper/2205.15716/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 24 references that are in the map.
- First posted on arXiv 31 May 2022.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [The Complexity of Decentralized Control of Markov Decision Processes](https://mlsanity.com/paper/1301.3836/) (2013)
- [Scientific multi-agent reinforcement learning for wall-models of turbulent flows](https://mlsanity.com/paper/2106.11144/) (2021)
- [Learning to Discretize: Solving 1D Scalar Conservation Laws via Deep Reinforcement Learning](https://mlsanity.com/paper/1905.11079/) (2019)
- [Backpropagation through Time and Space: Learning Numerical Methods with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2203.08937/) (2022)

## Abstract (arXiv)

Factored decentralized Markov decision process (Dec-MDP) is a framework for modeling sequential decision making problems in multi-agent systems. In this paper, we formalize the learning of numerical methods for hyperbolic partial differential equations (PDEs), specifically the Weighted Essentially Non-Oscillatory (WENO) scheme, as a factored Dec-MDP problem. We show that different reward formulations lead to either reinforcement learning (RL) or behavior cloning, and a homogeneous policy could be learned for all agents under the RL formulation with a policy gradient algorithm. Because the trained agents only act on their local observations, the multi-agent system can be used as a general numerical method for hyperbolic PDEs and generalize to different spatial discretizations, episode lengths, dimensions, and even equation types.