# PINN Balls: Scaling Second-Order Methods for PINNs with Domain Decomposition and Adaptive Sampling

arXiv: https://arxiv.org/abs/2510.21262 · HTML: https://mlsanity.com/paper/2510.21262/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts.
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 24 Oct 2025.

## Placed in

- Mixture-of-experts / [MoE theory and gating functions](https://mlsanity.com/moe/moe-theory-and-gating-functions/)

## Builds on (most cited first, 13 of 13)

- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition](https://mlsanity.com/paper/2003.05385/) (2020)
- [Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1801.06637/) (2018)
- [Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations](https://mlsanity.com/paper/2107.07871/) (2021)
- [Parallel Physics-Informed Neural Networks via Domain Decomposition](https://mlsanity.com/paper/2104.10013/) (2021)
- [Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodology](https://mlsanity.com/paper/2211.08939/) (2022)
- [Experience report of physics-informed neural networks in fluid simulations: pitfalls and frustration](https://mlsanity.com/paper/2205.14249/) (2022)
- [The Challenges of the Nonlinear Regime for Physics-Informed Neural Networks](https://mlsanity.com/paper/2402.03864/) (2024)
- [Adversarial Adaptive Sampling: Unify PINN and Optimal Transport for the Approximation of PDEs](https://mlsanity.com/paper/2305.18702/) (2023)
- [Meta Learning of Interface Conditions for Multi-Domain Physics-Informed Neural Networks](https://mlsanity.com/paper/2210.12669/) (2022)

## Abstract (arXiv)

Recent advances in Scientific Machine Learning have shown that second-order methods can enhance the training of Physics-Informed Neural Networks (PINNs), making them a suitable alternative to traditional numerical methods for Partial Differential Equations (PDEs). However, second-order methods induce large memory requirements, making them scale poorly with the model size. In this paper, we define a local Mixture of Experts (MoE) combining the parameter-efficiency of ensemble models and sparse coding to enable the use of second-order training. Our model -- textscPINN Balls -- also features a fully learnable domain decomposition structure, achieved through the use of Adversarial Adaptive Sampling (AAS), which adapts the DD to the PDE and its domain. textscPINN Balls achieves better accuracy than the state-of-the-art in scientific machine learning, while maintaining invaluable scalability properties and drawing from a sound theoretical background.