# Self-composing neural operators for high-frequency and multiscale PDE surrogates

arXiv: https://arxiv.org/abs/2508.20650 · HTML: https://mlsanity.com/paper/2508.20650/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 67 references that are in the map.
- First posted on arXiv 28 Aug 2025.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Fourier Neural Operator with Learned Deformations for PDEs on General Geometries](https://mlsanity.com/paper/2207.05209/) (2022)
- [Choose a Transformer: Fourier or Galerkin](https://mlsanity.com/paper/2105.14995/) (2021)
- [GNOT: A General Neural Operator Transformer for Operator Learning](https://mlsanity.com/paper/2302.14376/) (2023)
- [Multiwavelet-based Operator Learning for Differential Equations](https://mlsanity.com/paper/2109.13459/) (2021)
- [Convolutional Neural Operators for robust and accurate learning of PDEs](https://mlsanity.com/paper/2302.01178/) (2023)
- [Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations](https://mlsanity.com/paper/1710.10121/) (2017)
- [MIONet: Learning multiple-input operators via tensor product](https://mlsanity.com/paper/2202.06137/) (2022)
- [PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domain](https://mlsanity.com/paper/2004.13145/) (2020)
- [Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries](https://mlsanity.com/paper/2502.02414/) (2025)
- [Solving High-Dimensional PDEs with Latent Spectral Models](https://mlsanity.com/paper/2301.12664/) (2023)
- [JFB: Jacobian-Free Backpropagation for Implicit Networks](https://mlsanity.com/paper/2103.12803/) (2021)
- [On Training Implicit Models](https://mlsanity.com/paper/2111.05177/) (2021)
- [Blending Neural Operators and Relaxation Methods in PDE Numerical Solvers](https://mlsanity.com/paper/2208.13273/) (2022)
- [Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics](https://mlsanity.com/paper/2205.03990/) (2022)
- [Mitigating spectral bias for the multiscale operator learning](https://mlsanity.com/paper/2210.10890/) (2022)
- [MgNet: A Unified Framework of Multigrid and Convolutional Neural Network](https://mlsanity.com/paper/1901.10415/) (2019)
- [MgNO: Efficient Parameterization of Linear Operators via Multigrid](https://mlsanity.com/paper/2310.19809/) (2023)
- [SwitchNet: a neural network model for forward and inverse scattering problems](https://mlsanity.com/paper/1810.09675/) (2018)
- [Out-of-distributional risk bounds for neural operators with applications to the Helmholtz equation](https://mlsanity.com/paper/2301.11509/) (2023)
- [Multigrid-augmented deep learning preconditioners for the Helmholtz equation](https://mlsanity.com/paper/2203.11025/) (2022)
- [A Physics-Guided Neural Operator Learning Approach to Model Biological Tissues from Digital Image Correlation Measurements](https://mlsanity.com/paper/2204.00205/) (2022)
- [Deep Equilibrium Based Neural Operators for Steady-State PDEs](https://mlsanity.com/paper/2312.00234/) (2023)
- [IB-UQ: Information bottleneck based uncertainty quantification for neural function regression and neural operator learning](https://mlsanity.com/paper/2302.03271/) (2023)

## Built on by (most cited first, 3 of 3)

- [Mechanistic Interpretability with Sparse Autoencoder Neural Operators](https://mlsanity.com/paper/2509.03738/) (2025)
- [Neural Flow Operators can Approximate any Operator: Abstract Frameworks and Universal Approximations](https://mlsanity.com/paper/2605.22557/) (2026)
- [Starter-Iterator Neural Operator: A Unified Architecture for High-Fidelity Forward and Inverse PDE Problems](https://mlsanity.com/paper/2606.18305/) (2026)

## Abstract (arXiv)

Addressing the computational challenges of high-frequency and multiscale partial differential equations (PDEs), this work introduces a self-composing neural operator (SC-NO) framework. Inspired by classical fixed-point iterative solvers (e.g., multigrid, domain decomposition), the proposed architecture constructs a deep operator by repeatedly applying a single, parameter-efficient backbone block. This design mimics the update step of a numerical solver, allowing the model to progressively resolve complex solution features without increasing the parameter count. For practical training, we develop an adaptive ``Train-and-Unroll'' strategy that grows the composition depth during training, acting as a curriculum from shallow to deep self-composed models. We demonstrate the efficacy of this framework on the Helmholtz equation for ultrasound computed tomography (USCT), a problem characterized by high-frequency wave propagation in highly heterogeneous media. By instantiating the backbone with a multigrid-inspired architecture, the SC-NO effectively mitigates the spectral bias often observed in standard operator learning baselines. Numerical experiments show that our method reduces the prediction error significantly compared to Fourier Neural Operators (FNO) and their variants in the 300--500 kHz regime. Furthermore, we provide theoretical analysis linking the self-composition depth to approximation accuracy.