mlsanityData to 14 Oct 2026

From system models to class models: An in-context learning paradigm

arXiv 2308.13380 ↗First posted 25 Aug 2023eess.SY cs.LG cs.SYcited by 16

Not on the map. Why?

Abstract (arXiv)

Is it possible to understand the intricacies of a dynamical system not solely from its input/output pattern, but also by observing the behavior of other systems within the same class? This central question drives the study presented in this paper. In response to this query, we introduce a novel paradigm for system identification, addressing two primary tasks: one-step-ahead prediction and multi-step simulation. Unlike conventional methods, we do not directly estimate a model for the specific system. Instead, we learn a meta model that represents a class of dynamical systems. This meta model is trained on a potentially infinite stream of synthetic data, generated by simulators whose settings are randomly extracted from a probability distribution. When provided with a context from a new system-specifically, an input/output sequence-the meta model implicitly discerns its dynamics, enabling predictions of its behavior. The proposed approach harnesses the power of Transformers, renowned for their in-context learning capabilities. For one-step prediction, a GPT-like decoder-only architecture is utilized, whereas the simulation problem employs an encoder-decoder structure. Initial experimental results affirmatively answer our foundational question, opening doors to fresh research avenues in system identification.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 13 of its 28 references that are in the map. First posted on arXiv 25 Aug 2023.

Builds on

13 of 28 references.

References · 13

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  4. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
  5. 2018Meta-Learning Probabilistic Inference For Predictioncited by 110
  6. 2022General-Purpose In-Context Learning by Meta-Learning Transformerscited by 105
  7. 2020Continuous-time system identification with neural networks: Model structures and fitting criteriacited by 30
  8. 2020dynoNet: a neural network architecture for learning dynamical systemscited by 29
  9. 2019Performance-oriented model learning for data-driven MPC designcited by 22
  10. 2019Deep Convolutional Networks in System Identificationcited by 17
  11. 2023Can Transformers Learn Optimal Filtering for Unknown Systems?cited by 16
  12. 2022Deep Subspace Encoders for Nonlinear System Identificationcited by 13
1 more
  1. 2022Meta-Learning of Neural State-Space Models Using Data From Similar Systemscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 4; 2026: 8.

16 papers.

Sort:

By area:

  1. 2023On the adaptation of in-context learners for system identificationcited by 7
  2. 2024Enhanced Transformer architecture for in-context learning of dynamical systemscited by 7
  3. 2024The Duality of Generative AI and Reinforcement Learning in Robotics: A Reviewcited by 6
  4. 2025Meta-Learning for Physically-Constrained Neural System Identificationcited by 6
  5. 2024Synthetic data generation for system identification: leveraging knowledge transfer from similar systemscited by 5
  6. 2025Distributionally robust minimization in meta-learning for system identificationcited by 3
  7. 2025Manifold meta-learning for reduced-complexity neural system identificationcited by 2
  8. 2026Learning Nonlinear Systems In-Context: From Synthetic Data to Real-World Motor Controlcited by 1
  9. 2025LILAD: Learning In-context Lyapunov-stable Adaptive Dynamics Modelscited by 0
  10. 2026Diffusion Sequence Models for Generative In-Context Meta-Learning of Robot Dynamicscited by 0
  11. 2026Target Parameterization in Diffusion Models for Nonlinear Spatiotemporal System Identificationcited by 0
  12. 2026ASIA: an Autonomous System Identification Agentcited by 0
4 more
  1. 2026Learning Dynamical Systems from Multiple Sparse Datasets: A Hierarchical Bayesian Modeling Approachcited by 0
  2. 2026Attenuated in-context identification in time-series foundation models: diagnosis under counterfactual inputs and repair by synthetic forced-system fine-tuningcited by 0
  3. 2026In-Context Learning for System Identification: Probabilistic Formulation, Long-Sequence Handling, and Benchmark Validationcited by 0
  4. 2026Relational Learning as World Modelingcited by 0