mlsanityData to 7 Oct 2026

Bi-level Physics-Informed Neural Networks for PDE Constrained Optimization using Broyden's Hypergradients

arXiv 2209.07075 ↗First posted 15 Sep 2022cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

Deep learning based approaches like Physics-informed neural networks (PINNs) and DeepONets have shown promise on solving PDE constrained optimization (PDECO) problems. However, existing methods are insufficient to handle those PDE constraints that have a complicated or nonlinear dependency on optimization targets. In this paper, we present a novel bi-level optimization framework to resolve the challenge by decoupling the optimization of the targets and constraints. For the inner loop optimization, we adopt PINNs to solve the PDE constraints only. For the outer loop, we design a novel method by using Broyden's method based on the Implicit Function Theorem (IFT), which is efficient and accurate for approximating hypergradients. We further present theoretical explanations and error analysis of the hypergradients computation. Extensive experiments on multiple large-scale and nonlinear PDE constrained optimization problems demonstrate that our method achieves state-of-the-art results compared with strong baselines.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 21 of its 40 references that are in the map. First posted on arXiv 15 Sep 2022.

Builds on

21 of 40 references.

References · 21

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  3. 2021Learning the solution operator of parametric partial differential equations with physics-informed DeepOnetscited by 537
  4. 2019Meta-Learning with Implicit Gradientscited by 523
  5. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 357
  6. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 303
  7. 2021Physics-informed neural networks with hard constraints for inverse designcited by 277
  8. 2016Hyperparameter optimization with approximate gradientcited by 231
  9. 2020Bilevel Optimization: Convergence Analysis and Enhanced Designcited by 219
  10. 2018Truncated Back-propagation for Bilevel Optimizationcited by 187
  11. 2020On the Iteration Complexity of Hypergradient Computationcited by 154
  12. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
9 more
  1. 2020Multiscale Deep Equilibrium Modelscited by 149
  2. 2015Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameterscited by 71
  3. 2021Optimal control of PDEs using physics-informed neural networkscited by 51
  4. 2021Stability and Generalization of Bilevel Programming in Hyperparameter Optimizationcited by 31
  5. 2021Fast PDE-constrained optimization via self-supervised operator learningcited by 24
  6. 2020Greedy Quasi-Newton Methods with Explicit Superlinear Convergencecited by 21
  7. 2020Learning Composable Energy Surrogates for PDE Order Reductioncited by 13
  8. 2021Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiationcited by 7
  9. 2021Amortized Synthesis of Constrained Configurations Using a Differentiable Surrogatecited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 4; 2025: 4; 2026: 4.

17 papers.

Sort:

By area:

  1. 2024EraseAnything: Enabling Concept Erasure in Rectified Flow Transformerscited by 93
  2. 2022Physics-Informed Machine Learning: A Survey on Problems, Methods and Applicationscited by 88
  3. 2023NVFi: Neural Velocity Fields for 3D Physics Learning from Dynamic Videoscited by 28
  4. 2023Efficient PDE-Constrained optimization under high-dimensional uncertainty using derivative-informed neural operatorscited by 18
  5. 2024Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimizationcited by 11
  6. 2025FP64 is All You Need: Rethinking Failure Modes in Physics-Informed Neural Networkscited by 11
  7. 2025TRACE: Learning 3D Gaussian Physical Dynamics from Multi-view Videoscited by 11
  8. 2024Precise-Physics Driven Text-to-3D Generationcited by 9
  9. 2024BiLO: Bilevel Local Operator Learning for PDE Inverse Problemscited by 8
  10. 2025ParticleGS: Learning Neural Gaussian Particle Dynamics from Videos for Prior-free Physical Motion Extrapolationcited by 5
  11. 2023Accelerated primal-dual methods with enlarged step sizes and operator learning for nonsmooth optimal control problemscited by 4
  12. 2026Natural Hypergradient Descent: Algorithm Design, Convergence Analysis, and Parallel Implementationcited by 1
5 more
  1. 2023The ADMM-PINNs Algorithmic Framework for Nonsmooth PDE-Constrained Optimization: A Deep Learning Approachcited by 0
  2. 2025Conformal mapping based Physics-informed neural networks for designing neutral inclusionscited by 0
  3. 2026A Single-Loop Bilevel Deep Learning Method for Optimal Control of Obstacle Problemscited by 0
  4. 2026FlowSymm: Physics Aware, Symmetry Preserving Graph Attention for Network Flow Completioncited by 0
  5. 2026Harnessing AI for Inverse Partial Differential Equation Problems: Past, Present, and Prospectscited by 0