mlsanityData to 7 Oct 2026

Distributed Multigrid Neural Solvers on Megavoxel Domains

arXiv 2104.14538 ↗First posted 29 Apr 2021cs.LG stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

We consider the distributed training of large-scale neural networks that serve as PDE solvers producing full field outputs. We specifically consider neural solvers for the generalized 3D Poisson equation over megavoxel domains. A scalable framework is presented that integrates two distinct advances. First, we accelerate training a large model via a method analogous to the multigrid technique used in numerical linear algebra. Here, the network is trained using a hierarchy of increasing resolution inputs in sequence, analogous to the 'V', 'W', 'F', and 'Half-V' cycles used in multigrid approaches. In conjunction with the multi-grid approach, we implement a distributed deep learning framework which significantly reduces the time to solve. We show the scalability of this approach on both GPU (Azure VMs on Cloud) and CPU clusters (PSC Bridges2). This approach is deployed to train a generalized 3D Poisson solver that scales well to predict output full-field solutions up to the resolution of 512x512x512 for a high dimensional family of inputs.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 24 of its 54 references that are in the map. First posted on arXiv 29 Apr 2021.

Builds on

24 of 54 references.

References · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  4. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 674
  5. 2017Solving high-dimensional partial differential equations using deep learningcited by 549
  6. 2021Physics-informed neural networks with hard constraints for inverse designcited by 277
  7. 2018Horovod: fast and easy distributed deep learning in TensorFlowcited by 275
  8. 2019Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Datacited by 261
  9. 2020hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decompositioncited by 236
  10. 2017Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equationscited by 208
  11. 2018Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantificationcited by 199
  12. 2018Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysiscited by 154
12 more
  1. 2018Physics-Informed Generative Adversarial Networks for Stochastic Differential Equationscited by 133
  2. 2016Accelerating Eulerian Fluid Simulation With Convolutional Networkscited by 128
  3. 2019An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applicationscited by 118
  4. 2020NVIDIA SimNet^{TM}: an AI-accelerated multi-physics simulation frameworkcited by 58
  5. 2020Learning Algebraic Multigrid Using Graph Neural Networkscited by 54
  6. 2019A Multigrid Method for Efficiently Training Video Modelscited by 52
  7. 2020DiscretizationNet: A Machine-Learning based solver for Navier-Stokes Equations using Finite Volume Discretizationcited by 35
  8. 2021Learning optimal multigrid smoothers via neural networkscited by 17
  9. 2019Highly-scalable, physics-informed GANs for learning solutions of stochastic PDEscited by 11
  10. 2019Solving Irregular and Data-enriched Differential Equations using Deep Neural Networkscited by 5
  11. 2020Deep Generative Models that Solve PDEs: Distributed Computing for Training Large Data-Free Modelscited by 5
  12. 2021Translating Numerical Concepts for PDEs into Neural Architecturescited by 4

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 3; 2023: 1.

5 papers.

  1. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 603
  2. 2023Geometry-Informed Neural Operator for Large-Scale 3D PDEscited by 293
  3. 2021NeuFENet: Neural Finite Element Solutions with Theoretical Bounds for Parametric PDEscited by 10
  4. 2022Neural PDE Solvers for Irregular Domainscited by 6
  5. 2022SciAI4Industry -- Solving PDEs for industry-scale problems with deep learningcited by 2