# Synthetic data generation for system identification: leveraging knowledge transfer from similar systems

arXiv: https://arxiv.org/abs/2403.05164 · HTML: https://mlsanity.com/paper/2403.05164/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 14 references that are in the map.
- First posted on arXiv 8 Mar 2024.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Synthetic Data for Deep Learning](https://mlsanity.com/paper/1909.11512/) (2019)
- [dynoNet: a neural network architecture for learning dynamical systems](https://mlsanity.com/paper/2006.02250/) (2020)
- [From system models to class models: An in-context learning paradigm](https://mlsanity.com/paper/2308.13380/) (2023)
- [On the adaptation of in-context learners for system identification](https://mlsanity.com/paper/2312.04083/) (2023)

## Built on by (most cited first, 5 of 5)

- [Enhanced Transformer architecture for in-context learning of dynamical systems](https://mlsanity.com/paper/2410.03291/) (2024)
- [Fine Tuning a Simulation-Driven Estimator](https://mlsanity.com/paper/2504.04480/) (2025)
- [Learning Nonlinear Systems In-Context: From Synthetic Data to Real-World Motor Control](https://mlsanity.com/paper/2602.07173/) (2026)
- [Diffusion Sequence Models for Generative In-Context Meta-Learning of Robot Dynamics](https://mlsanity.com/paper/2604.13366/) (2026)
- [In-Context Learning for System Identification: Probabilistic Formulation, Long-Sequence Handling, and Benchmark Validation](https://mlsanity.com/paper/2610.13411/) (2026)

## Abstract (arXiv)

This paper addresses the challenge of overfitting in the learning of dynamical systems by introducing a novel approach for the generation of synthetic data, aimed at enhancing model generalization and robustness in scenarios characterized by data scarcity. Central to the proposed methodology is the concept of knowledge transfer from systems within the same class. Specifically, synthetic data is generated through a pre-trained meta-model that describes a broad class of systems to which the system of interest is assumed to belong. Training data serves a dual purpose: firstly, as input to the pre-trained meta model to discern the system's dynamics, enabling the prediction of its behavior and thereby generating synthetic output sequences for new input sequences; secondly, in conjunction with synthetic data, to define the loss function used for model estimation. A validation dataset is used to tune a scalar hyper-parameter balancing the relative importance of training and synthetic data in the definition of the loss function. The same validation set can be also used for other purposes, such as early stopping during the training, fundamental to avoid overfitting in case of small-size training datasets. The efficacy of the approach is shown through a numerical example that highlights the advantages of integrating synthetic data into the system identification process.