# Deep Equilibrium Based Neural Operators for Steady-State PDEs

arXiv: https://arxiv.org/abs/2312.00234 · HTML: https://mlsanity.com/paper/2312.00234/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 41 references that are in the map.
- First posted on arXiv 30 Nov 2023.

## Builds on (most cited first, 25 of 25)

- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Fourier Neural Operator with Learned Deformations for PDEs on General Geometries](https://mlsanity.com/paper/2207.05209/) (2022)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [Message Passing Neural PDE Solvers](https://mlsanity.com/paper/2202.03376/) (2022)
- [GNOT: A General Neural Operator Transformer for Operator Learning](https://mlsanity.com/paper/2302.14376/) (2023)
- [Transformer for Partial Differential Equations' Operator Learning](https://mlsanity.com/paper/2205.13671/) (2022)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Factorized Fourier Neural Operators](https://mlsanity.com/paper/2111.13802/) (2021)
- [Monotone operator equilibrium networks](https://mlsanity.com/paper/2006.08591/) (2020)
- [JFB: Jacobian-Free Backpropagation for Implicit Networks](https://mlsanity.com/paper/2103.12803/) (2021)
- [Learning Neural PDE Solvers with Convergence Guarantees](https://mlsanity.com/paper/1906.01200/) (2019)
- [On Training Implicit Models](https://mlsanity.com/paper/2111.05177/) (2021)
- [Reviving and Improving Recurrent Back-Propagation](https://mlsanity.com/paper/1803.06396/) (2018)
- [A physics-informed operator regression framework for extracting data-driven continuum models](https://mlsanity.com/paper/2009.11992/) (2020)
- [Lipschitz Bounded Equilibrium Networks](https://mlsanity.com/paper/2010.01732/) (2020)
- [Learning to correct spectral methods for simulating turbulent flows](https://mlsanity.com/paper/2207.00556/) (2022)
- [Mitigating spectral bias for the multiscale operator learning](https://mlsanity.com/paper/2210.10890/) (2022)
- [Path Independent Equilibrium Models Can Better Exploit Test-Time Computation](https://mlsanity.com/paper/2211.09961/) (2022)
- [On the Representation of Solutions to Elliptic PDEs in Barron Spaces](https://mlsanity.com/paper/2106.07539/) (2021)
- [Parametric Complexity Bounds for Approximating PDEs with Neural Networks](https://mlsanity.com/paper/2103.02138/) (2021)
- [Neural Network Approximations of PDEs Beyond Linearity: A Representational Perspective](https://mlsanity.com/paper/2210.12101/) (2022)

## Built on by (most cited first, 19 of 19)

- [CodePDE: An Inference Framework for LLM-driven PDE Solver Generation](https://mlsanity.com/paper/2505.08783/) (2025)
- [On the Benefits of Memory for Modeling Time-Dependent PDEs](https://mlsanity.com/paper/2409.02313/) (2024)
- [UPS: Efficiently Building Foundation Models for PDE Solving via Cross-Modal Adaptation](https://mlsanity.com/paper/2403.07187/) (2024)
- [Toward a Better Understanding of Fourier Neural Operators from a Spectral Perspective](https://mlsanity.com/paper/2404.07200/) (2024)
- [HyPINO: Multi-Physics Neural Operators via HyperPINNs and the Method of Manufactured Solutions](https://mlsanity.com/paper/2509.05117/) (2025)
- [Generative Neural Operators of Log-Complexity Can Simultaneously Solve Infinitely Many Convex Programs](https://mlsanity.com/paper/2508.14995/) (2025)
- [Quantitative Approximation for Neural Operators in Nonlinear Parabolic Equations](https://mlsanity.com/paper/2410.02151/) (2024)
- [Self-composing neural operators for high-frequency and multiscale PDE surrogates](https://mlsanity.com/paper/2508.20650/) (2025)
- [Polynomial Scaling is Possible For Neural Operator Approximations of Structured Families of BSDEs](https://mlsanity.com/paper/2410.14788/) (2024)
- [Certified Robustness for Deep Equilibrium Models via Serialized Random Smoothing](https://mlsanity.com/paper/2411.00899/) (2024)
- [Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approach](https://mlsanity.com/paper/2602.03297/) (2026)
- [Bifurcation Models: Learning Set-Valued Solution Maps with Weight-Tied Dynamics](https://mlsanity.com/paper/2605.07277/) (2026)
- [The Kuramoto Neural Operator: Learning to Solve PDEs via Coupled Oscillator Dynamics](https://mlsanity.com/paper/2608.10234/) (2026)
- [Response Renormalization for Critical Deep Equilibrium Models](https://mlsanity.com/paper/2608.23725/) (2026)
- [NewPINNs: Physics-Informing Neural Networks Using Conventional Solvers for Partial Differential Equations](https://mlsanity.com/paper/2601.17207/) (2026)
- [Generalization Error Bounds for Picard-Type Operator Learning in Nonlinear Parabolic PDEs](https://mlsanity.com/paper/2605.10277/) (2026)
- [Drifting Models for Surrogate Flow Modeling](https://mlsanity.com/paper/2606.07481/) (2026)
- [Why Directly Learning Periodic Trajectories Can Fail](https://mlsanity.com/paper/2609.32254/) (2026)
- [Beyond Simulation: Retain-and-Repair Neural Operators for Real-World Adaptation](https://mlsanity.com/paper/2609.39387/) (2026)

## Abstract (arXiv)

Data-driven machine learning approaches are being increasingly used to solve partial differential equations (PDEs). They have shown particularly striking successes when training an operator, which takes as input a PDE in some family, and outputs its solution. However, the architectural design space, especially given structural knowledge of the PDE family of interest, is still poorly understood. We seek to remedy this gap by studying the benefits of weight-tied neural network architectures for steady-state PDEs. To achieve this, we first demonstrate that the solution of most steady-state PDEs can be expressed as a fixed point of a non-linear operator. Motivated by this observation, we propose FNO-DEQ, a deep equilibrium variant of the FNO architecture that directly solves for the solution of a steady-state PDE as the infinite-depth fixed point of an implicit operator layer using a black-box root solver and differentiates analytically through this fixed point resulting in O(1) training memory. Our experiments indicate that FNO-DEQ-based architectures outperform FNO-based baselines with 4× the number of parameters in predicting the solution to steady-state PDEs such as Darcy Flow and steady-state incompressible Navier-Stokes. Finally, we show FNO-DEQ is more robust when trained with datasets with more noisy observations than the FNO-based baselines, demonstrating the benefits of using appropriate inductive biases in architectural design for different neural network based PDE solvers. Further, we show a universal approximation result that demonstrates that FNO-DEQ can approximate the solution to any steady-state PDE that can be written as a fixed point equation.