# Deep transfer operator learning for partial differential equations under conditional shift

arXiv: https://arxiv.org/abs/2204.09810 · HTML: https://mlsanity.com/paper/2204.09810/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 43 references that are in the map.
- First posted on arXiv 20 Apr 2022.

## Builds on (most cited first, 14 of 14)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [What is being transferred in transfer learning?](https://mlsanity.com/paper/2008.11687/) (2020)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [Transfer learning enhanced physics informed neural network for phase-field modeling of fracture](https://mlsanity.com/paper/1907.02531/) (2019)
- [One-Shot Transfer Learning of Physics-Informed Neural Networks](https://mlsanity.com/paper/2110.11286/) (2021)
- [Mosaic Flows: A Transferable Deep Learning Framework for Solving PDEs on Unseen Domains](https://mlsanity.com/paper/2104.10873/) (2021)
- [On the influence of over-parameterization in manifold based surrogates and deep neural operators](https://mlsanity.com/paper/2203.05071/) (2022)
- [Hypothesis Transfer Learning via Transformation Functions](https://mlsanity.com/paper/1612.01020/) (2016)
- [Machine learning materials physics: Multi-resolution neural networks learn the free energy and nonlinear elastic response of evolving microstructures](https://mlsanity.com/paper/2001.01575/) (2019)

## Built on by (most cited first, 30 of 83)

- [Transolver: A Fast Transformer Solver for PDEs on General Geometries](https://mlsanity.com/paper/2402.02366/) (2024)
- [Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior](https://mlsanity.com/paper/2306.00258/) (2023)
- [In-Context Operator Learning with Data Prompts for Differential Equation Problems](https://mlsanity.com/paper/2304.07993/) (2023)
- [Multiple Physics Pretraining for Physical Surrogate Models](https://mlsanity.com/paper/2310.02994/) (2023)
- [Blending Neural Operators and Relaxation Methods in PDE Numerical Solvers](https://mlsanity.com/paper/2208.13273/) (2022)
- [Walrus: A Cross-Domain Foundation Model for Continuum Dynamics](https://mlsanity.com/paper/2511.15684/) (2025)
- [Transformers as Neural Operators for Solutions of Differential Equations with Finite Regularity](https://mlsanity.com/paper/2405.19166/) (2024)
- [Neural Operator Learning for Long-Time Integration in Dynamical Systems with Recurrent Neural Networks](https://mlsanity.com/paper/2303.02243/) (2023)
- [PDE-Transformer: Efficient and Versatile Transformers for Physics Simulations](https://mlsanity.com/paper/2505.24717/) (2025)
- [On the training and generalization of deep operator networks](https://mlsanity.com/paper/2309.01020/) (2023)
- [PDE Generalization of In-Context Operator Networks: A Study on 1D Scalar Nonlinear Conservation Laws](https://mlsanity.com/paper/2401.07364/) (2024)
- [Learning stiff chemical kinetics using extended deep neural operators](https://mlsanity.com/paper/2302.12645/) (2023)
- [L-HYDRA: Multi-Head Physics-Informed Neural Networks](https://mlsanity.com/paper/2301.02152/) (2023)
- [Stacked networks improve physics-informed training: applications to neural networks and deep operator networks](https://mlsanity.com/paper/2311.06483/) (2023)
- [Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flow](https://mlsanity.com/paper/2206.03198/) (2022)
- [Learning in latent spaces improves the predictive accuracy of deep neural operators](https://mlsanity.com/paper/2304.07599/) (2023)
- [Synergistic Learning with Multi-Task DeepONet for Efficient PDE Problem Solving](https://mlsanity.com/paper/2408.02198/) (2024)
- [Basis-to-Basis Operator Learning Using Function Encoders](https://mlsanity.com/paper/2410.00171/) (2024)
- [Diffeomorphism Neural Operator for various domains and parameters of partial differential equations](https://mlsanity.com/paper/2402.12475/) (2024)
- [Masked Autoencoders are PDE Learners](https://mlsanity.com/paper/2403.17728/) (2024)
- [Strategies for Pretraining Neural Operators](https://mlsanity.com/paper/2406.08473/) (2024)
- [MORPH: PDE Foundation Models with Arbitrary Data Modality](https://mlsanity.com/paper/2509.21670/) (2025)
- [P2C2Net: PDE-Preserved Coarse Correction Network for efficient prediction of spatiotemporal dynamics](https://mlsanity.com/paper/2411.00040/) (2024)
- [Sound propagation in realistic interactive 3D scenes with parameterized sources using deep neural operators](https://mlsanity.com/paper/2308.05141/) (2023)
- [Active operator learning with predictive uncertainty quantification for partial differential equations](https://mlsanity.com/paper/2503.03178/) (2025)
- [Stochastic Latent Transformer: Efficient Modelling of Stochastically Forced Zonal Jets](https://mlsanity.com/paper/2310.16741/) (2023)
- [Towards a Physics Foundation Model](https://mlsanity.com/paper/2509.13805/) (2025)
- [Variational operator learning: A unified paradigm marrying training neural operators and solving partial differential equations](https://mlsanity.com/paper/2304.04234/) (2023)
- [Towards Size-Independent Generalization Bounds for Deep Operator Nets](https://mlsanity.com/paper/2205.11359/) (2022)
- [A hybrid Decoder-DeepONet operator regression framework for unaligned observation data](https://mlsanity.com/paper/2308.09274/) (2023)

## Abstract (arXiv)

Transfer learning (TL) enables the transfer of knowledge gained in learning to perform one task (source) to a related but different task (target), hence addressing the expense of data acquisition and labeling, potential computational power limitations, and dataset distribution mismatches. We propose a new TL framework for task-specific learning (functional regression in partial differential equations (PDEs)) under conditional shift based on the deep operator network (DeepONet). Task-specific operator learning is accomplished by fine-tuning task-specific layers of the target DeepONet using a hybrid loss function that allows for the matching of individual target samples while also preserving the global properties of the conditional distribution of target data. Inspired by the conditional embedding operator theory, we minimize the statistical distance between labeled target data and the surrogate prediction on unlabeled target data by embedding conditional distributions onto a reproducing kernel Hilbert space. We demonstrate the advantages of our approach for various TL scenarios involving nonlinear PDEs under diverse conditions due to shift in the geometric domain and model dynamics. Our TL framework enables fast and efficient learning of heterogeneous tasks despite significant differences between the source and target domains.