# From system models to class models: An in-context learning paradigm

arXiv: https://arxiv.org/abs/2308.13380 · HTML: https://mlsanity.com/paper/2308.13380/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 25 Aug 2023.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [What Can Transformers Learn In-Context? A Case Study of Simple Function Classes](https://mlsanity.com/paper/2208.01066/) (2022)
- [Meta-Learning Probabilistic Inference For Prediction](https://mlsanity.com/paper/1805.09921/) (2018)
- [General-Purpose In-Context Learning by Meta-Learning Transformers](https://mlsanity.com/paper/2212.04458/) (2022)
- [Continuous-time system identification with neural networks: Model structures and fitting criteria](https://mlsanity.com/paper/2006.02915/) (2020)
- [dynoNet: a neural network architecture for learning dynamical systems](https://mlsanity.com/paper/2006.02250/) (2020)
- [Performance-oriented model learning for data-driven MPC design](https://mlsanity.com/paper/1904.10839/) (2019)
- [Deep Convolutional Networks in System Identification](https://mlsanity.com/paper/1909.01730/) (2019)
- [Can Transformers Learn Optimal Filtering for Unknown Systems?](https://mlsanity.com/paper/2308.08536/) (2023)
- [Deep Subspace Encoders for Nonlinear System Identification](https://mlsanity.com/paper/2210.14816/) (2022)
- [Meta-Learning of Neural State-Space Models Using Data From Similar Systems](https://mlsanity.com/paper/2211.07768/) (2022)

## Built on by (most cited first, 16 of 16)

- [On the adaptation of in-context learners for system identification](https://mlsanity.com/paper/2312.04083/) (2023)
- [Enhanced Transformer architecture for in-context learning of dynamical systems](https://mlsanity.com/paper/2410.03291/) (2024)
- [The Duality of Generative AI and Reinforcement Learning in Robotics: A Review](https://mlsanity.com/paper/2410.16411/) (2024)
- [Meta-Learning for Physically-Constrained Neural System Identification](https://mlsanity.com/paper/2501.06167/) (2025)
- [Synthetic data generation for system identification: leveraging knowledge transfer from similar systems](https://mlsanity.com/paper/2403.05164/) (2024)
- [Distributionally robust minimization in meta-learning for system identification](https://mlsanity.com/paper/2506.18074/) (2025)
- [Manifold meta-learning for reduced-complexity neural system identification](https://mlsanity.com/paper/2504.11811/) (2025)
- [Learning Nonlinear Systems In-Context: From Synthetic Data to Real-World Motor Control](https://mlsanity.com/paper/2602.07173/) (2026)
- [LILAD: Learning In-context Lyapunov-stable Adaptive Dynamics Models](https://mlsanity.com/paper/2511.21846/) (2025)
- [Diffusion Sequence Models for Generative In-Context Meta-Learning of Robot Dynamics](https://mlsanity.com/paper/2604.13366/) (2026)
- [Target Parameterization in Diffusion Models for Nonlinear Spatiotemporal System Identification](https://mlsanity.com/paper/2604.17566/) (2026)
- [ASIA: an Autonomous System Identification Agent](https://mlsanity.com/paper/2605.10480/) (2026)
- [Learning Dynamical Systems from Multiple Sparse Datasets: A Hierarchical Bayesian Modeling Approach](https://mlsanity.com/paper/2606.24966/) (2026)
- [Attenuated in-context identification in time-series foundation models: diagnosis under counterfactual inputs and repair by synthetic forced-system fine-tuning](https://mlsanity.com/paper/2610.08118/) (2026)
- [In-Context Learning for System Identification: Probabilistic Formulation, Long-Sequence Handling, and Benchmark Validation](https://mlsanity.com/paper/2610.13411/) (2026)
- [Relational Learning as World Modeling](https://mlsanity.com/paper/2610.16763/) (2026)

## Abstract (arXiv)

Is it possible to understand the intricacies of a dynamical system not solely from its input/output pattern, but also by observing the behavior of other systems within the same class? This central question drives the study presented in this paper. In response to this query, we introduce a novel paradigm for system identification, addressing two primary tasks: one-step-ahead prediction and multi-step simulation. Unlike conventional methods, we do not directly estimate a model for the specific system. Instead, we learn a meta model that represents a class of dynamical systems. This meta model is trained on a potentially infinite stream of synthetic data, generated by simulators whose settings are randomly extracted from a probability distribution. When provided with a context from a new system-specifically, an input/output sequence-the meta model implicitly discerns its dynamics, enabling predictions of its behavior. The proposed approach harnesses the power of Transformers, renowned for their in-context learning capabilities. For one-step prediction, a GPT-like decoder-only architecture is utilized, whereas the simulation problem employs an encoder-decoder structure. Initial experimental results affirmatively answer our foundational question, opening doors to fresh research avenues in system identification.