mlsanityData to 14 Oct 2026

PDE Generalization of In-Context Operator Networks: A Study on 1D Scalar Nonlinear Conservation Laws

arXiv 2401.07364 ↗First posted 14 Jan 2024cs.LG cs.AI cs.NAcited by 31

Not on the map. Why?

Abstract (arXiv)

Can we build a single large model for a wide range of PDE-related scientific learning tasks? Can this model generalize to new PDEs, even of new forms, without any fine-tuning? In-context operator learning and the corresponding model In-Context Operator Networks (ICON) represent an initial exploration of these questions. The capability of ICON regarding the first question has been demonstrated previously. In this paper, we present a detailed methodology for solving PDE problems with ICON, and show how a single ICON model can make forward and reverse predictions for different equations with different strides, provided with appropriately designed data prompts. We show the positive evidence to the second question, i.e., ICON can generalize well to some PDEs with new forms without any fine-tuning. This is exemplified through a study on 1D scalar nonlinear conservation laws, a family of PDEs with temporal evolution. We also show how to broaden the range of problems that an ICON model can address, by transforming functions and equations to ICON's capability scope. We believe that the progress in this paper is a significant step towards the goal of training a foundation model for PDE-related tasks under the in-context operator learning framework.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Builds on 44 of its 75 references that are in the map. First posted on arXiv 14 Jan 2024.

Builds on

44 of 75 references.

References · 44

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  5. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  6. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  7. 2020Learning to Simulate Complex Physics with Graph Networkscited by 880
  8. 2020Learning Mesh-Based Simulation with Graph Networkscited by 868
  9. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
  10. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 678
  11. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 583
  12. 2017Solving high-dimensional partial differential equations using deep learningcited by 556
32 more
  1. 2021Learning the solution operator of parametric partial differential equations with physics-informed DeepOnetscited by 540
  2. 2021Physics-Informed Neural Operator for Learning Partial Differential Equationscited by 527
  3. 2021Machine learning accelerated computational fluid dynamicscited by 471
  4. 2022Message Passing Neural PDE Solverscited by 386
  5. 2017PDE-Net: Learning PDEs from Datacited by 311
  6. 2020Model Reduction and Neural Networks for Parametric PDEscited by 258
  7. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  8. 2018Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantificationcited by 199
  9. 2017Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equationscited by 191
  10. 2023Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behaviorcited by 131
  11. 2019Transfer learning enhanced physics informed neural network for phase-field modeling of fracturecited by 116
  12. 2023In-Context Operator Learning with Data Prompts for Differential Equation Problemscited by 101
  13. 2023Multiple Physics Pretraining for Physical Surrogate Modelscited by 98
  14. 2019A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problemscited by 97
  15. 2022Learning Operators with Coupled Attentioncited by 90
  16. 2022Deep transfer operator learning for partial differential equations under conditional shiftcited by 83
  17. 2022Structured Prompting: Scaling In-Context Learning to 1,000 Examplescited by 81
  18. 2022Parallel Context Windows for Large Language Modelscited by 81
  19. 2020Transfer learning based multi-fidelity physics informed deep neural networkcited by 68
  20. 2022Reliable extrapolation of deep neural operators informed by physics or sparse observationscited by 62
  21. 2021One-Shot Transfer Learning of Physics-Informed Neural Networkscited by 49
  22. 2022Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarioscited by 46
  23. 2021Mosaic Flows: A Transferable Deep Learning Framework for Solving PDEs on Unseen Domainscited by 37
  24. 2020Alternating the Population and Control Neural Networks to Solve High-Dimensional Stochastic Mean-Field Gamescited by 35
  25. 2019Potential Flow Generator with L2 Optimal Transport Regularity for Generative Modelscited by 31
  26. 2022Transfer Learning Enhanced DeepONet for Long-Time Prediction of Evolution Equationscited by 28
  27. 2022Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flowcited by 20
  28. 2023PROSE: Predicting Operators and Symbolic Expressions using Multimodal Transformerscited by 16
  29. 2022SVD-PINNs: Transfer Learning of Physics-Informed Neural Networks via Singular Value Decompositioncited by 13
  30. 2020Analysis of three dimensional potential problems in non-homogeneous media with physics-informed deep collocation method using material transfer learning and sensitivity analysiscited by 11
  31. 2021Unsupervised Reservoir Computing for Solving Ordinary Differential Equationscited by 11
  32. 2023Multi-fidelity prediction of fluid flow and temperature field based on transfer learning using Fourier Neural Operatorcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 11; 2025: 9; 2026: 11.

31 papers.

Sort:

By area:

  1. 2024Towards a Foundation Model for Partial Differential Equations: Multi-Operator Learning and Extrapolationcited by 46
  2. 2024PROSE-FD: A Multimodal PDE Foundation Model for Learning Multiple Operators for Forecasting Fluid Dynamicscited by 34
  3. 2024VICON: Vision In-Context Operator Networks for Multi-Physics Fluid Dynamics Predictioncited by 25
  4. 2025DISCO: learning to DISCover an evolution Operator for multi-physics-agnostic predictioncited by 21
  5. 2024FMint: Bridging Human Designed and Data Pretrained Models for Differential Equation Foundation Modelcited by 19
  6. 2024Nonlocal Attention Operator: Materializing Hidden Knowledge Towards Interpretable Physics Discoverycited by 18
  7. 2025BCAT: A Block Causal Transformer for PDE Foundation Models for Fluid Dynamicscited by 18
  8. 2024MODNO: Multi Operator Learning With Distributed Neural Operatorscited by 14
  9. 2024LeMON: Learning to Learn Multi-Operator Networkscited by 14
  10. 2024Model Balancing Helps Low-data Training and Fine-tuningcited by 11
  11. 2025A Multimodal PDE Foundation Model for Prediction and Scientific Text Descriptionscited by 10
  12. 2025Probabilistic operator learning: generative modeling and uncertainty quantification for foundation models of differential equationscited by 10
19 more
  1. 2024Time-Series Forecasting, Knowledge Distillation, and Refinement within a Multimodal PDE Foundation Modelcited by 7
  2. 2025PI-MFM: Physics-informed multimodal foundation model for solving partial differential equationscited by 7
  3. 2024In-Context Learning of Linear Systems: Generalization Theory and Applications to Operator Learningcited by 6
  4. 2025Spectral Embedding via Chebyshev Bases for Robust DeepONet Approximationcited by 6
  5. 2025Continuum Transformers Perform In-Context Learning by Operator Gradient Descentcited by 3
  6. 2026Graph In-Context Operator Networks for Generalizable Spatiotemporal Predictioncited by 3
  7. 2025Neural Interpretable PDEs: Harmonizing Fourier Insights with Attention for Scalable and Interpretable Physics Discoverycited by 2
  8. 2026Evolving Ensemble of Agentscited by 2
  9. 2026Spectral Audit of In-Context Operator Networkscited by 2
  10. 2026Chain of Operators: An Inference-Time Harness for In-Context Operator Learningcited by 2
  11. 2026Generalized Neural Operator for Parametric and Boundary-Value Problemscited by 1
  12. 2024MaD-Scientist: AI-based Scientist solving Convection-Diffusion-Reaction Equations Using Massive PINN-Based Prior Datacited by 0
  13. 2025In-Context Multi-Operator Learning with DeepOSetscited by 0
  14. 2026Generalization Limits of In-Context Operator Networks for Higher-Order Partial Differential Equationscited by 0
  15. 2026A Robust In-Context Model for Conservation Laws: Injecting Context into Flux Neural Operators via Recurrent Vision Transformerscited by 0
  16. 2026Operator Learning for Reconstructing Flow Fields from Sparse Measurements: a Language Model Approachcited by 0
  17. 2026A foundation model of numerical intelligence with cross-disciplinary generalizationcited by 0
  18. 2026Transferability of Learned States in Neural PDE Solverscited by 0
  19. 2026PDE-PFN: Prior-Data Fitted Neural PDE Solvercited by 0