# Accelerating Data Generation for Neural Operators via Krylov Subspace Recycling

arXiv: https://arxiv.org/abs/2401.09516 · HTML: https://mlsanity.com/paper/2401.09516/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 64 references that are in the map.
- First posted on arXiv 17 Jan 2024.

## Builds on (most cited first, 14 of 14)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flow](https://mlsanity.com/paper/2109.03697/) (2021)
- [U-NO: U-shaped Neural Operators](https://mlsanity.com/paper/2204.11127/) (2022)
- [Clifford Neural Layers for PDE Modeling](https://mlsanity.com/paper/2209.04934/) (2022)
- [Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications](https://mlsanity.com/paper/2211.08064/) (2022)
- [Learning to Optimize Multigrid PDE Solvers](https://mlsanity.com/paper/1902.10248/) (2019)
- [Learning Neural PDE Solvers with Convergence Guarantees](https://mlsanity.com/paper/1906.01200/) (2019)
- [Blending Neural Operators and Relaxation Methods in PDE Numerical Solvers](https://mlsanity.com/paper/2208.13273/) (2022)
- [INO: Invariant Neural Operators for Learning Complex Physical Systems with Momentum Conservation](https://mlsanity.com/paper/2212.14365/) (2022)
- [A neural operator-based surrogate solver for free-form electromagnetic inverse design](https://mlsanity.com/paper/2302.01934/) (2023)
- [Weakly-Supervised Deep Learning of Heat Transport via Physics Informed Loss](https://mlsanity.com/paper/1807.11374/) (2018)

## Built on by (most cited first, 12 of 12)

- [Active Learning for Neural PDE Solvers](https://mlsanity.com/paper/2408.01536/) (2024)
- [Accelerating PDE Data Generation via Differential Operator Action in Solution Space](https://mlsanity.com/paper/2402.05957/) (2024)
- [Mixture-of-Experts Operator Transformer for Large-Scale PDE Pre-Training](https://mlsanity.com/paper/2510.25803/) (2025)
- [Learning PDE Solvers with Physics and Data: A Unifying View of Physics-Informed Neural Networks and Neural Operators](https://mlsanity.com/paper/2601.14517/) (2026)
- [SymMaP: Improving Computational Efficiency in Linear Solvers through Symbolic Preconditioning](https://mlsanity.com/paper/2510.24170/) (2025)
- [STNet: Spectral Transformation Network for Solving Operator Eigenvalue Problem](https://mlsanity.com/paper/2510.23986/) (2025)
- [Accelerating Data Generation for Nonlinear temporal PDEs via homologous perturbation in solution space](https://mlsanity.com/paper/2510.21592/) (2025)
- [Accelerating Eigenvalue Dataset Generation via Chebyshev Subspace Filter](https://mlsanity.com/paper/2510.23215/) (2025)
- [Learning Neural Operators from Partial Observations via Latent Autoregressive Modeling](https://mlsanity.com/paper/2601.15547/) (2026)
- [Accelerating IC Thermal Simulation Data Generation via Block Krylov and Operator Action](https://mlsanity.com/paper/2510.23221/) (2025)
- [Learning-Guided Integration Contours Construction for Fast Large-Scale Generalized Eigensolvers](https://mlsanity.com/paper/2511.01927/) (2025)
- [HGATSolver: A Heterogeneous Graph Attention Solver for Fluid-Structure Interaction](https://mlsanity.com/paper/2601.09251/) (2026)

## Abstract (arXiv)

Learning neural operators for solving partial differential equations (PDEs) has attracted great attention due to its high inference efficiency. However, training such operators requires generating a substantial amount of labeled data, i.e., PDE problems together with their solutions. The data generation process is exceptionally time-consuming, as it involves solving numerous systems of linear equations to obtain numerical solutions to the PDEs. Many existing methods solve these systems independently without considering their inherent similarities, resulting in extremely redundant computations. To tackle this problem, we propose a novel method, namely Sorting Krylov Recycling (SKR), to boost the efficiency of solving these systems, thus significantly accelerating data generation for neural operators training. To the best of our knowledge, SKR is the first attempt to address the time-consuming nature of data generation for learning neural operators. The working horse of SKR is Krylov subspace recycling, a powerful technique for solving a series of interrelated systems by leveraging their inherent similarities. Specifically, SKR employs a sorting algorithm to arrange these systems in a sequence, where adjacent systems exhibit high similarities. Then it equips a solver with Krylov subspace recycling to solve the systems sequentially instead of independently, thus effectively enhancing the solving efficiency. Both theoretical analysis and extensive experiments demonstrate that SKR can significantly accelerate neural operator data generation, achieving a remarkable speedup of up to 13.9 times.