M2M: Learning controllable Multi of experts and multi-scale operators are the Partial Differential Equations need
Not on the map. Why?
Abstract (arXiv)
Learning the evolutionary dynamics of Partial Differential Equations (PDEs) is critical in understanding dynamic systems, yet current methods insufficiently learn their representations. This is largely due to the multi-scale nature of the solution, where certain regions exhibit rapid oscillations while others evolve more slowly. This paper introduces a framework of multi-scale and multi-expert (M2M) neural operators designed to simulate and learn PDEs efficiently. We employ a divide-and-conquer strategy to train a multi-expert gated network for the dynamic router policy. Our method incorporates a controllable prior gating mechanism that determines the selection rights of experts, enhancing the model's efficiency. To optimize the learning process, we have implemented a PI (Proportional, Integral) control strategy to adjust the allocation rules precisely. This universal controllable approach allows the model to achieve greater accuracy. We test our approach on benchmark 2D Navier-Stokes equations and provide a custom multi-scale dataset. M2M can achieve higher simulation accuracy and offer improved interpretability compared to baseline methods.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 38 of its 59 references that are in the map. First posted on arXiv 1 Oct 2024.
Builds on
38 of 59 references.
References · 38
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
- 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
- 2020Learning to Simulate Complex Physics with Graph Networkscited by 880
- 2020Learning Mesh-Based Simulation with Graph Networkscited by 868
- 2022Message Passing Neural PDE Solverscited by 386
- 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 368
- 2017Snapshot Ensembles: Train 1, get M for freecited by 350
- 2021Choose a Transformer: Fourier or Galerkincited by 343
- 2023GNOT: A General Neural Operator Transformer for Operator Learningcited by 336
- 2023Geometry-Informed Neural Operator for Large-Scale 3D PDEscited by 303
26 more
- 2021U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flowcited by 297
- 2021Ensemble deep learning: A reviewcited by 284
- 2023Spherical Fourier Neural Operators: Learning Stable Dynamics on the Spherecited by 233
- 2023Neural Operators for Accelerating Scientific Simulations and Designcited by 221
- 2023Convolutional Neural Operators for robust and accurate learning of PDEscited by 214
- 2020Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solverscited by 205
- 2019Towards Physics-informed Deep Learning for Turbulent Flow Predictioncited by 204
- 2021Enhancing Computational Fluid Dynamics with Machine Learningcited by 138
- 2019Machine learning for molecular simulationcited by 110
- 2020Transformers for Modeling Physical Systemscited by 106
- 2017Visual Interaction Networkscited by 95
- 2024Operator Learning: Algorithms and Analysiscited by 85
- 2020Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domainscited by 79
- 2024Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEscited by 59
- 2022Mitigating spectral bias for the multiscale operator learningcited by 44
- 2022Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodologycited by 44
- 2022Learning to Accelerate Partial Differential Equations via Latent Global Evolutioncited by 39
- 2023Learning Neural PDE Solvers with Parameter-Guided Channel Attentioncited by 38
- 2023Domain Agnostic Fourier Neural Operatorscited by 38
- 2023Koopman neural operator as a mesh-free solver of non-linear partial differential equationscited by 29
- 2023Physics informed WNOcited by 27
- 2024BENO: Boundary-embedded Neural Operators for Elliptic PDEscited by 24
- 2024Scaling physics-informed hard constraints with mixture-of-expertscited by 24
- 2023Neural Spectral Methods: Self-supervised learning in the spectral domaincited by 14
- 2020A Tailored Convolutional Neural Network for Nonlinear Manifold Learning of Computational Physics Data using Unstructured Spatial Discretizationscited by 6
- 2021A Divide-and-Conquer Algorithm for Distributed Optimization on Networkscited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.