# Scalable Mechanistic Neural Networks for Differential Equations and Machine Learning

arXiv: https://arxiv.org/abs/2410.06074 · HTML: https://mlsanity.com/paper/2410.06074/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 33 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Builds on (most cited first, 20 of 20)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Universal Differential Equations for Scientific Machine Learning](https://mlsanity.com/paper/2001.04385/) (2020)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [Weakly-Supervised Disentanglement Without Compromises](https://mlsanity.com/paper/2002.02886/) (2020)
- [Neural Operators for Accelerating Scientific Simulations and Design](https://mlsanity.com/paper/2309.15325/) (2023)
- [PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers](https://mlsanity.com/paper/2308.05732/) (2023)
- [SINDy-PI: A Robust Algorithm for Parallel Implicit Sparse Identification of Nonlinear Dynamics](https://mlsanity.com/paper/2004.02322/) (2020)
- [ClimODE: Climate and Weather Forecasting with Physics-informed Neural ODEs](https://mlsanity.com/paper/2404.10024/) (2024)
- [Lie Point Symmetry Data Augmentation for Neural PDE Solvers](https://mlsanity.com/paper/2202.07643/) (2022)
- [A Mathematical Guide to Operator Learning](https://mlsanity.com/paper/2312.14688/) (2023)
- [On Second Order Behaviour in Augmented Neural ODEs](https://mlsanity.com/paper/2006.07220/) (2020)
- [ODEFormer: Symbolic Regression of Dynamical Systems with Transformers](https://mlsanity.com/paper/2310.05573/) (2023)
- ["Hey, that's not an ODE": Faster ODE Adjoints via Seminorms](https://mlsanity.com/paper/2009.09457/) (2020)
- [Discovering Sparse Interpretable Dynamics from Partial Observations](https://mlsanity.com/paper/2107.10879/) (2021)
- [Marrying Causal Representation Learning with Dynamical Systems for Science](https://mlsanity.com/paper/2405.13888/) (2024)
- [Mechanistic Neural Networks for Scientific Machine Learning](https://mlsanity.com/paper/2402.13077/) (2024)

## Built on by (most cited first, 5 of 5)

- [Discovering Physics Laws of Dynamical Systems via Invariant Function Learning](https://mlsanity.com/paper/2502.04495/) (2025)
- [Adaptive Physics-informed Neural Networks: A Survey](https://mlsanity.com/paper/2503.18181/) (2025)
- [Mechanistic PDE Networks for Discovery of Governing Equations](https://mlsanity.com/paper/2502.18377/) (2025)
- [Learning Explicit Single-Cell Dynamics Using ODE Representations](https://mlsanity.com/paper/2510.02903/) (2025)
- [Identifying ODEs from Unstructured Data with Causal Representation Learning](https://mlsanity.com/paper/2609.37083/) (2026)

## Abstract (arXiv)

We propose Scalable Mechanistic Neural Network (S-MNN), an enhanced neural network framework designed for scientific machine learning applications involving long temporal sequences. By reformulating the original Mechanistic Neural Network (MNN) (Pervez et al., 2024), we reduce the computational time and space complexities from cubic and quadratic with respect to the sequence length, respectively, to linear. This significant improvement enables efficient modeling of long-term dynamics without sacrificing accuracy or interpretability. Extensive experiments demonstrate that S-MNN matches the original MNN in precision while substantially reducing computational resources. Consequently, S-MNN can drop-in replace the original MNN in applications, providing a practical and efficient tool for integrating mechanistic bottlenecks into neural network models of complex dynamical systems. Source code is available at https://github.com/IST-DASLab/ScalableMNN.