# Neuro-Spectral Architectures for Causal Physics-Informed Networks

arXiv: https://arxiv.org/abs/2509.04966 · HTML: https://mlsanity.com/paper/2509.04966/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 77 references that are in the map.
- First posted on arXiv 5 Sep 2025.

## Builds on (most cited first, 30 of 33)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks](https://mlsanity.com/paper/1901.06523/) (2019)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks](https://mlsanity.com/paper/2207.10289/) (2022)
- [How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks](https://mlsanity.com/paper/2009.11848/) (2020)
- [Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems](https://mlsanity.com/paper/2111.02801/) (2021)
- [Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networks](https://mlsanity.com/paper/1805.12076/) (2018)
- [An Expert's Guide to Training Physics-informed Neural Networks](https://mlsanity.com/paper/2308.08468/) (2023)
- [ClimODE: Climate and Weather Forecasting with Physics-informed Neural ODEs](https://mlsanity.com/paper/2404.10024/) (2024)
- [Physics informed deep learning for computational elastodynamics without labeled data](https://mlsanity.com/paper/2006.08472/) (2020)
- [Trivializations for Gradient-Based Optimization on Manifolds](https://mlsanity.com/paper/1909.09501/) (2019)
- [PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networks](https://mlsanity.com/paper/2307.11833/) (2023)
- [Learning in Sinusoidal Spaces with Physics-Informed Neural Networks](https://mlsanity.com/paper/2109.09338/) (2021)
- [Reliable extrapolation of deep neural operators informed by physics or sparse observations](https://mlsanity.com/paper/2212.06347/) (2022)
- [DPM: A Novel Training Method for Physics-Informed Neural Networks in Extrapolation](https://mlsanity.com/paper/2012.02681/) (2020)
- [Quadratic Residual Networks: A New Class of Neural Networks for Solving Forward and Inverse Problems in Physics Involving PDEs](https://mlsanity.com/paper/2101.08366/) (2021)
- [PINNACLE: PINN Adaptive ColLocation and Experimental points selection](https://mlsanity.com/paper/2404.07662/) (2024)
- [RoPINN: Region Optimized Physics-Informed Neural Networks](https://mlsanity.com/paper/2405.14369/) (2024)
- [PINNup: Robust neural network wavefield solutions using frequency upscaling and neuron splitting](https://mlsanity.com/paper/2109.14536/) (2021)
- [Neural Spectral Methods: Self-supervised learning in the spectral domain](https://mlsanity.com/paper/2312.05225/) (2023)

## Built on by (most cited first, 3 of 3)

- [FLOWING: Implicit Neural Flows for Structure-Preserving Morphing](https://mlsanity.com/paper/2510.09537/) (2025)
- [Functional Gradient Descent with Adaptive Representations](https://mlsanity.com/paper/2606.16926/) (2026)
- [Oscillatory State-Space Models as Inductive Biases for Physics-Informed Neural PDE Solvers](https://mlsanity.com/paper/2606.02623/) (2026)

## Abstract (arXiv)

Physics-Informed Neural Networks (PINNs) have emerged as a powerful framework for solving partial differential equations (PDEs). However, standard MLP-based PINNs often fail to converge when dealing with complex initial value problems, leading to solutions that violate causality and suffer from a spectral bias towards low-frequency components. To address these issues, we introduce NeuSA (Neuro-Spectral Architectures), a novel class of PINNs inspired by classical spectral methods, designed to solve linear and nonlinear PDEs with variable coefficients. NeuSA learns a projection of the underlying PDE onto a spectral basis, leading to a finite-dimensional representation of the dynamics which is then integrated with an adapted Neural ODE (NODE). This allows us to overcome spectral bias, by leveraging the high-frequency components enabled by the spectral representation; to enforce causality, by inheriting the causal structure of NODEs, and to start training near the target solution, by means of an initialization scheme based on classical methods. We validate NeuSA on canonical benchmarks for linear and nonlinear wave equations, demonstrating strong performance as compared to other architectures, with faster convergence, improved temporal consistency and superior predictive accuracy. Code and pretrained models are available in https://github.com/arthur-bizzi/neusa.