mlsanityData to 14 Oct 2026

SineNet: Learning Temporal Dynamics in Time-Dependent Partial Differential Equations

arXiv 2403.19507 ↗First posted 28 Mar 2024cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

We consider using deep neural networks to solve time-dependent partial differential equations (PDEs), where multi-scale processing is crucial for modeling complex, time-evolving dynamics. While the U-Net architecture with skip connections is commonly used by prior studies to enable multi-scale processing, our analysis shows that the need for features to evolve across layers results in temporally misaligned features in skip connections, which limits the model's performance. To address this limitation, we propose SineNet, consisting of multiple sequentially connected U-shaped network blocks, referred to as waves. In SineNet, high-resolution features are evolved progressively through multiple stages, thereby reducing the amount of misalignment within each stage. We furthermore analyze the role of skip connections in enabling both parallel and sequential processing of multi-scale information. Our method is rigorously tested on multiple PDE datasets, including the Navier-Stokes equations and shallow water equations, showcasing the advantages of our proposed approach over conventional U-Nets with a comparable parameter budget. We further demonstrate that increasing the number of waves in SineNet while maintaining the same number of parameters leads to a monotonically improved performance. The results highlight the effectiveness of SineNet and the potential of our approach in advancing the state-of-the-art in neural PDE solver design. Our code is available as part of AIRS (https://github.com/divelab/AIRS).

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 45 of its 59 references that are in the map. First posted on arXiv 28 Mar 2024.

Builds on

45 of 59 references.

References · 45

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  6. 2016Layer Normalizationcited by 5,669
  7. 2018Neural Ordinary Differential Equationscited by 3,779
  8. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  9. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  10. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  11. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  12. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
33 more
  1. 2017Deformable Convolutional Networkscited by 1,488
  2. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 915
  3. 2020Learning to Simulate Complex Physics with Graph Networkscited by 880
  4. 2020Learning Mesh-Based Simulation with Graph Networkscited by 868
  5. 2021Learning the solution operator of parametric partial differential equations with physics-informed DeepOnetscited by 540
  6. 2021Physics-Informed Neural Operator for Learning Partial Differential Equationscited by 527
  7. 2019Graph U-Netscited by 492
  8. 2021Machine learning accelerated computational fluid dynamicscited by 471
  9. 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 427
  10. 2021Neural Operator: Learning Maps Between Function Spacescited by 414
  11. 2022PDEBENCH: An Extensive Benchmark for Scientific Machine Learningcited by 413
  12. 2022Message Passing Neural PDE Solverscited by 386
  13. 2020Multipole Graph Neural Operator for Parametric Partial Differential Equationscited by 349
  14. 2021Factorized Fourier Neural Operatorscited by 249
  15. 2022U-NO: U-shaped Neural Operatorscited by 230
  16. 2023PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solverscited by 216
  17. 2020Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solverscited by 205
  18. 2019Towards Physics-informed Deep Learning for Turbulent Flow Predictioncited by 204
  19. 2022Towards Multi-spatiotemporal-scale Generalized PDE Modelingcited by 197
  20. 2020Learning to Control PDEs with Differentiable Physicscited by 132
  21. 2023Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systemscited by 121
  22. 2021Learned Coarse Models for Efficient Turbulence Simulationcited by 91
  23. 2017W-Net: A Deep Model for Fully Unsupervised Image Segmentationcited by 57
  24. 2022Generalizing to New Physical Systems via Context-Informed Dynamics Modelcited by 50
  25. 2022Learning to correct spectral methods for simulating turbulent flowscited by 49
  26. 2023Group Equivariant Fourier Neural Operators for Partial Differential Equationscited by 45
  27. 2021Meta-Learning Dynamics Forecasting Using Task Inferencecited by 38
  28. 2022Learning the Dynamics of Physical Systems from Sparse Observations with Finite Element Networkscited by 34
  29. 2022Physics-Embedded Neural Networks: Graph Neural PDE Solvers with Mixed Boundary Conditionscited by 33
  30. 2018LadderNet: Multi-path networks based on U-Net for medical image segmentationcited by 29
  31. 2022Transform Once: Efficient Operator Learning in Frequency Domaincited by 20
  32. 2018Stacked U-Nets: A No-Frills Approach to Natural Image Segmentationcited by 9
  33. 2023MetaPhysiCa: OOD Robustness in Physics-informed Machine Learningcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 5; 2026: 5.

14 papers.

Sort:

By area:

  1. 2024Recent Advances on Machine Learning for Computational Fluid Dynamics: A Surveycited by 28
  2. 2024Active Learning for Neural PDE Solverscited by 22
  3. 2024What You See is Not What You Get: Neural Partial Differential Equations and The Illusion of Learningcited by 5
  4. 2025Overtone: Cyclic Patch Modulation for Clean, Efficient, and Flexible Physics Emulatorscited by 5
  5. 2025A Two-Phase Deep Learning Framework for Adaptive Time-Stepping in High-Speed Flow Modelingcited by 3
  6. 2024PIC2O-Sim: A Physics-Inspired Causality-Aware Dynamic Convolutional Neural Operator for Ultra-Fast Photonic Device FDTD Simulationcited by 2
  7. 2025MAPS: Multi-Fidelity AI-Augmented Photonic Simulation and Inverse Design Infrastructurecited by 2
  8. 2025Hierarchical-embedding autoencoder with a predictor (HEAP) as efficient architecture for learning long-term evolution of complex multi-scale physical systemscited by 2
  9. 2026System-Anchored Knee Estimation for Low-Cost Context Window Selection in PDE Forecastingcited by 1
  10. 2025Learning and Transferring Physical Models through Derivativescited by 0
  11. 2026Orbital Transformers for Predicting Wavefunctions in Time-Dependent Density Functional Theorycited by 0
  12. 2026Deep Wave Network for Modeling Multi-Scale Physical Dynamicscited by 0
2 more
  1. 2026LEIA: Learned Environment for Interactive Architected Materialscited by 0
  2. 2026From Simulated Fluids to Living Tissue: Zero-Shot Foundation Model Transfer to Perturbed Embryoscited by 0