S2GPT-PINNs: Sparse and Small models for PDEs
Not on the map. Why?
Abstract (arXiv)
We propose S2GPT-PINN, a sparse and small model for solving parametric partial differential equations (PDEs). Similar to Small Language Models (SLMs), S2GPT-PINN is tailored to domain-specific (families of) PDEs and characterized by its compact architecture and minimal computational power. Leveraging a small amount of extremely high quality data via a mathematically rigorous greedy algorithm that is enabled by the large full-order models, S2GPT-PINN relies on orders of magnitude less parameters than PINNs to achieve extremely high efficiency via two levels of customizations. The first is knowledge distillation via task-specific activation functions that are transferred from Pre-Trained PINNs. The second is a judicious down-sampling when calculating the physics-informed loss of the network compressing the number of data sites by orders of magnitude to the size of the small model.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 19 of its 35 references that are in the map. First posted on arXiv 25 May 2025.
Builds on
19 of 35 references.
References · 19
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
- 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
- 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 669
- 2017Solving high-dimensional partial differential equations using deep learningcited by 556
- 2024Small Language Models: Survey, Measurements, and Insightscited by 112
- 2020A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEscited by 105
- 2021POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decompositioncited by 79
7 more
- 2022Failure-informed adaptive sampling for PINNscited by 64
- 2021Meta-Auto-Decoder for Solving Parametric Partial Differential Equationscited by 36
- 2024PINNACLE: PINN Adaptive ColLocation and Experimental points selectioncited by 32
- 2024Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLMcited by 27
- 2021Galerkin Neural Networks: A Framework for Approximating Variational Equations with Error Controlcited by 24
- 2023GPT-PINN: Generative Pre-Trained Physics-Informed Neural Networks toward non-intrusive Meta-learning of parametric PDEscited by 20
- 2024TGPT-PINN: Nonlinear model reduction with transformed GPT-PINNscited by 5
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Tensor-Compressed and Fully-Quantized Training of Neural PDE Solvers
- 2025PINN Balls: Scaling Second-Order Methods for PINNs with Domain Decomposition and Adaptive Sampling
- 2025Lang-PINN: From Language to Physics-Informed Neural Networks via a Multi-Agent Framework
- 2025KD-PINN: Knowledge-Distilled PINNs for ultra-low-latency real-time neural PDE solvers
- 2025Sub-Sequential Physics-Informed Learning with State Space Model
- 2023LatentPINNs: Generative physics-informed neural networks via a latent representation learning
- 2026Unlearning Noise in PINNs: A Selective Pruning Framework for PDE Inverse Problems
- 2026DGNet: Discrete Green Networks for Data-Efficient Learning of Spatiotemporal PDEs