Accelerating Legacy Numerical Solvers by Non-intrusive Gradient-based Meta-solving
Not on the map. Why?
Abstract (arXiv)
Scientific computing is an essential tool for scientific discovery and engineering design, and its computational cost is always a main concern in practice. To accelerate scientific computing, it is a promising approach to use machine learning (especially meta-learning) techniques for selecting hyperparameters of traditional numerical methods. There have been numerous proposals to this direction, but many of them require automatic-differentiable numerical methods. However, in reality, many practical applications still depend on well-established but non-automatic-differentiable legacy codes, which prevents practitioners from applying the state-of-the-art research to their own problems. To resolve this problem, we propose a non-intrusive methodology with a novel gradient estimation technique to combine machine learning and legacy numerical codes without any modification. We theoretically and numerically show the advantage of the proposed method over other baselines and present applications of accelerating established non-automatic-differentiable numerical solvers implemented in PETSc, a widely used open-source numerical software library.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 25 of its 47 references that are in the map. First posted on arXiv 5 May 2024.
Builds on
25 of 47 references.
References · 25
- 2016Gaussian Error Linear Units (GELUs)cited by 3,502
- 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
- 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 950
- 2015Automatic differentiation in machine learning: a surveycited by 855
- 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 608
- 2022PDEBENCH: An Extensive Benchmark for Scientific Machine Learningcited by 413
- 2021Learning to Optimize: A Primer and A Benchmarkcited by 208
- 2020Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solverscited by 205
- 2020Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systemscited by 164
- 2017Backpropagation through the Void: Optimizing control variates for black-box gradient estimationcited by 106
- 2019Learning Neural PDE Solvers with Convergence Guaranteescited by 77
- 2022Gradients without Backpropagationcited by 73
13 more
- 2021Physics-based Deep Learningcited by 68
- 2022Scaling Forward Gradient With Local Lossescited by 65
- 2020Learning Algebraic Multigrid Using Graph Neural Networkscited by 54
- 2023Can Forward Gradient Match Backpropagation?cited by 26
- 2019Poisson CNN: Convolutional neural networks for the solution of the Poisson equation on a Cartesian meshcited by 22
- 2019Deep Learning of Preconditioners for Conjugate Gradient Solvers in Urban Water Related Problemscited by 20
- 2022Optimization without Backpropagationcited by 11
- 2021Neural Fixed-Point Acceleration for Convex Optimizationcited by 10
- 2021Personalized Algorithm Generation: A Case Study in Learning ODE Integratorscited by 9
- 2022Low-Variance Forward Gradients using Direct Feedback Alignment and Momentumcited by 7
- 2022Principled Acceleration of Iterative Numerical Methods Using Machine Learningcited by 6
- 2022Neural-network preconditioners for solving the Dirac equation in lattice gauge theorycited by 6
- 2019Neural network gradient-based learning of black-box function interfacescited by 5
Built on by
Citing papers per year: 2026: 2.
2 papers.