# Advancing Generalization in PINNs through Latent-Space Representations

arXiv: https://arxiv.org/abs/2411.19125 · HTML: https://mlsanity.com/paper/2411.19125/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 68 references that are in the map.
- First posted on arXiv 28 Nov 2024.

## Builds on (most cited first, 30 of 42)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [Message Passing Neural PDE Solvers](https://mlsanity.com/paper/2202.03376/) (2022)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [How to train your neural ODE: the world of Jacobian and kinetic regularization](https://mlsanity.com/paper/2002.02798/) (2020)
- [Respecting causality is all you need for training physics-informed neural networks](https://mlsanity.com/paper/2203.07404/) (2022)
- [Dynamical Variational Autoencoders: A Comprehensive Review](https://mlsanity.com/paper/2008.12595/) (2020)
- [A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networks](https://mlsanity.com/paper/2406.02917/) (2024)
- [Long-time integration of parametric evolution equations with physics-informed DeepONets](https://mlsanity.com/paper/2106.05384/) (2021)
- [Operator Learning with Neural Fields: Tackling PDEs on General Geometries](https://mlsanity.com/paper/2306.07266/) (2023)
- [DeepOKAN: Deep Operator Network Based on Kolmogorov Arnold Networks for Mechanics Problems](https://mlsanity.com/paper/2405.19143/) (2024)
- [Learning to Optimize Multigrid PDE Solvers](https://mlsanity.com/paper/1902.10248/) (2019)
- [Continuous PDE Dynamics Forecasting with Implicit Neural Representations](https://mlsanity.com/paper/2209.14855/) (2022)
- [Residual-based attention in physics-informed neural networks](https://mlsanity.com/paper/2307.00379/) (2023)
- [Separable Physics-Informed Neural Networks](https://mlsanity.com/paper/2306.15969/) (2023)
- [CROM: Continuous Reduced-Order Modeling of PDEs Using Implicit Neural Representations](https://mlsanity.com/paper/2206.02607/) (2022)
- [Learning to Solve PDE-constrained Inverse Problems with Graph Networks](https://mlsanity.com/paper/2206.00711/) (2022)
- [Learning Dynamical Systems from Partial Observations](https://mlsanity.com/paper/1902.11136/) (2019)
- [Meta-Auto-Decoder for Solving Parametric Partial Differential Equations](https://mlsanity.com/paper/2111.08823/) (2021)
- [Transformers as Neural Operators for Solutions of Differential Equations with Finite Regularity](https://mlsanity.com/paper/2405.19166/) (2024)
- [Nonlinear Reconstruction for Operator Learning of PDEs with Discontinuities](https://mlsanity.com/paper/2210.01074/) (2022)

## Built on by (most cited first, 5 of 5)

- [Towards a Foundation Model for Partial Differential Equations Across Physics Domains](https://mlsanity.com/paper/2511.21861/) (2025)
- [Parameterized Temporal Extrapolation of PDEs via Disentangled Latent Dynamics Manifold Fusion](https://mlsanity.com/paper/2603.12676/) (2026)
- [Limitations of Physics-Informed Neural Networks: a Study on Smart Grid Surrogation](https://mlsanity.com/paper/2508.21559/) (2025)
- [Variational Matrix-Learning Fourier Networks for Parametric Multiphysics Surrogates](https://mlsanity.com/paper/2605.02280/) (2026)
- [Meta-Inverse Physics-Informed Neural Networks for High-Dimensional Ordinary Differential Equations](https://mlsanity.com/paper/2605.03511/) (2026)

## Abstract (arXiv)

Physics-informed neural networks (PINNs) have made significant strides in modeling dynamical systems governed by partial differential equations (PDEs). However, their generalization capabilities across varying scenarios remain limited. To overcome this limitation, we propose PIDO, a novel physics-informed neural PDE solver designed to generalize effectively across diverse PDE configurations, including varying initial conditions, PDE coefficients, and training time horizons. PIDO exploits the shared underlying structure of dynamical systems with different properties by projecting PDE solutions into a latent space using auto-decoding. It then learns the dynamics of these latent representations, conditioned on the PDE coefficients. Despite its promise, integrating latent dynamics models within a physics-informed framework poses challenges due to the optimization difficulties associated with physics-informed losses. To address these challenges, we introduce a novel approach that diagnoses and mitigates these issues within the latent space. This strategy employs straightforward yet effective regularization techniques, enhancing both the temporal extrapolation performance and the training stability of PIDO. We validate PIDO on a range of benchmarks, including 1D combined equations and 2D Navier-Stokes equations. Additionally, we demonstrate the transferability of its learned representations to downstream applications such as long-term integration and inverse problems.