# Adaptive parameters identification for nonlinear dynamics using deep permutation invariant networks

arXiv: https://arxiv.org/abs/2501.11350 · HTML: https://mlsanity.com/paper/2501.11350/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 20 Jan 2025.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network](https://mlsanity.com/paper/1812.04426/) (2018)
- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [Physics-informed learning of governing equations from scarce data](https://mlsanity.com/paper/2005.03448/) (2020)
- [Sparse identification of nonlinear dynamics for model predictive control in the low-data limit](https://mlsanity.com/paper/1711.05501/) (2017)
- [Dynamic mode decomposition with control](https://mlsanity.com/paper/1409.6358/) (2014)
- [Extrapolation and learning equations](https://mlsanity.com/paper/1610.02995/) (2016)
- [Interpretable Scientific Discovery with Symbolic Regression: A Review](https://mlsanity.com/paper/2211.10873/) (2022)
- [ODEFormer: Symbolic Regression of Dynamical Systems with Transformers](https://mlsanity.com/paper/2310.05573/) (2023)
- [Forecast Evaluation for Data Scientists: Common Pitfalls and Best Practices](https://mlsanity.com/paper/2203.10716/) (2022)
- [Any equation is a forest: Symbolic genetic algorithm for discovering open-form partial differential equations (SGA-PDE)](https://mlsanity.com/paper/2106.11927/) (2021)
- [A Unified Framework for Sparse Relaxed Regularized Regression: SR3](https://mlsanity.com/paper/1807.05411/) (2018)
- [Predicting Ordinary Differential Equations with Transformers](https://mlsanity.com/paper/2307.12617/) (2023)
- [A Review of Data-Driven Discovery for Dynamic Systems](https://mlsanity.com/paper/2210.10663/) (2022)
- [SINDy with Control: A Tutorial](https://mlsanity.com/paper/2108.13404/) (2021)

## Built on by (most cited first, 5 of 5)

- [Data-driven Discovery of Digital Twins in Biomedical Research](https://mlsanity.com/paper/2508.21484/) (2025)
- [Interpretable neural network system identification method for two families of second-order systems based on characteristic curves](https://mlsanity.com/paper/2509.10632/) (2025)
- [Hybrid Adaptive Modeling in Process Monitoring: Leveraging Sequence Encoders and Physics-Informed Neural Networks](https://mlsanity.com/paper/2505.14252/) (2025)
- [Physics-Informed Neural Networks and Sequence Encoder: Application to heating and early cooling of thermo-stamping process](https://mlsanity.com/paper/2603.26245/) (2026)
- [Optimal Transport-based Permutation-Invariant Bayesian Optimization of Offshore Wind Farm Layouts](https://mlsanity.com/paper/2606.00009/) (2026)

## Abstract (arXiv)

The promising outcomes of dynamical system identification techniques, such as SINDy [Brunton et al. 2016], highlight their advantages in providing qualitative interpretability and extrapolation compared to non-interpretable deep neural networks [Rudin 2019]. These techniques suffer from parameter updating in real-time use cases, especially when the system parameters are likely to change during or between processes. Recently, the OASIS [Bhadriraju et al. 2020] framework introduced a data-driven technique to address the limitations of real-time dynamical system parameters updating, yielding interesting results. Nevertheless, we show in this work that superior performance can be achieved using more advanced model architectures. We present an innovative encoding approach, based mainly on the use of Set Encoding methods of sequence data, which give accurate adaptive model identification for complex dynamic systems, with variable input time series length. Two Set Encoding methods are used, the first is Deep Set [Zaheer et al. 2017], and the second is Set Transformer [Lee et al. 2019]. Comparing Set Transformer to OASIS framework on Lotka Volterra for real-time local dynamical system identification and time series forecasting, we find that the Set Transformer architecture is well adapted to learning relationships within data sets. We then compare the two Set Encoding methods based on the Lorenz system for online global dynamical system identification. Finally, we trained a Deep Set model to perform identification and characterization of abnormalities for 1D heat-transfer problem.