# Generalizing Dynamics Modeling More Easily from Representation Perspective

arXiv: https://arxiv.org/abs/2603.22655 · HTML: https://mlsanity.com/paper/2603.22655/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 64 references that are in the map.
- First posted on arXiv 24 Mar 2026.

## Builds on (most cited first, 30 of 30)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [Neural Relational Inference for Interacting Systems](https://mlsanity.com/paper/1802.04687/) (2018)
- [Spatial-Temporal Graph ODE Networks for Traffic Flow Forecasting](https://mlsanity.com/paper/2106.12931/) (2021)
- [LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters](https://mlsanity.com/paper/2308.08469/) (2023)
- [A Survey of Knowledge Enhanced Pre-trained Language Models](https://mlsanity.com/paper/2211.05994/) (2022)
- [Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs](https://mlsanity.com/paper/2202.08408/) (2022)
- [Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecasting](https://mlsanity.com/paper/2502.04395/) (2025)
- [Learning Continuous System Dynamics from Irregularly-Sampled Partial Observations](https://mlsanity.com/paper/2011.03880/) (2020)
- [A Survey on Time-Series Pre-Trained Models](https://mlsanity.com/paper/2305.10716/) (2023)
- [Generalizing to New Physical Systems via Context-Informed Dynamics Model](https://mlsanity.com/paper/2202.01889/) (2022)
- [Physics-Guided Deep Learning for Dynamical Systems: A Survey](https://mlsanity.com/paper/2107.01272/) (2021)
- [LEADS: Learning Dynamical Systems that Generalize Across Environments](https://mlsanity.com/paper/2106.04546/) (2021)
- [Neural Dynamics on Complex Networks](https://mlsanity.com/paper/1908.06491/) (2019)
- [Meta-Learning Dynamics Forecasting Using Task Inference](https://mlsanity.com/paper/2102.10271/) (2021)
- [A Comprehensive Survey of Regression Based Loss Functions for Time Series Forecasting](https://mlsanity.com/paper/2211.02989/) (2022)
- [A probabilistic model for the numerical solution of initial value problems](https://mlsanity.com/paper/1610.05261/) (2016)
- [Generalizing Graph ODE for Learning Complex System Dynamics across Environments](https://mlsanity.com/paper/2307.04287/) (2023)
- [LangTime: A Language-Guided Unified Model for Time Series Forecasting with Proximal Policy Optimization](https://mlsanity.com/paper/2503.08271/) (2025)
- [Climate Modeling with Neural Diffusion Equations](https://mlsanity.com/paper/2111.06011/) (2021)
- [Generative Flows with Invertible Attentions](https://mlsanity.com/paper/2106.03959/) (2021)
- [Physics-Informed Regularization for Domain-Agnostic Dynamical System Modeling](https://mlsanity.com/paper/2410.06366/) (2024)
- [Causal Graph ODE: Continuous Treatment Effect Modeling in Multi-agent Dynamical Systems](https://mlsanity.com/paper/2403.00178/) (2024)
- [PGODE: Towards High-quality System Dynamics Modeling](https://mlsanity.com/paper/2311.06554/) (2023)

## Abstract (arXiv)

Learning system dynamics from observations is a critical problem in many applications over various real-world complex systems, e.g., climate, ecology, and fluid systems. Recently, neural dynamics modeling method have become a prevalent solution that embeds the object's observations into a latent space before learning dynamics using neural methods such as neural Ordinary Differential Equations (ODE). Existing dynamics modeling methods induce a specific model for each observation of different complex systems, resulting in poor generalization across systems. Inspired by the great success of pre-trained models, we conduct a generalized Pre-trained Dynamics EncoDER (PDEDER) which can embed the original state observations into a latent space where the dynamics can be captured more easily. To conduct the generalized PDEDER, we pre-train any Pre-trained Language Model (PLM) by minimizing the Lyapunov exponent objective, which constrains the chaotic behavior of governing dynamics learned in the latent space. By penalizing the divergence of embedded observations, our PDEDER promotes locally stable and well-structured latent dynamics, thereby facilitating more effective dynamics modeling than in the original observation space. In addition, we incorporate reconstruction and forecasting objectives to mitigate the risk of obtaining an over-smoothed latent space. Specifically, we collect 152 sets of real-world and synthetic observations from 23 complex systems as pre-training corpora and employ them to pre-train PDEDER. Given any future dynamic observation, we can fine-tune PDEDER with any specific dynamics modeling method. We evaluate PDEDER on 12 dynamic systems by short/long-term forecasting under both in-domain and cross-domain settings, and the empirical results indicate the effectiveness and generalizability of PDEDER.