# Iterative self-transfer learning: A general methodology for response time-history prediction based on small dataset

arXiv: https://arxiv.org/abs/2306.08700 · HTML: https://mlsanity.com/paper/2306.08700/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 64 references that are in the map.
- First posted on arXiv 14 Jun 2023.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [A Survey on Deep Transfer Learning](https://mlsanity.com/paper/1808.01974/) (2018)
- [ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation](https://mlsanity.com/paper/2107.02137/) (2021)
- [Simultaneous Deep Transfer Across Domains and Tasks](https://mlsanity.com/paper/1510.02192/) (2015)
- [MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification](https://mlsanity.com/paper/2004.12239/) (2020)
- [Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1705.00609/) (2017)
- [Transfer learning enhanced physics informed neural network for phase-field modeling of fracture](https://mlsanity.com/paper/1907.02531/) (2019)
- [Transfer Learning with Dynamic Adversarial Adaptation Network](https://mlsanity.com/paper/1909.08184/) (2019)
- [Deep Transfer Network: Unsupervised Domain Adaptation](https://mlsanity.com/paper/1503.00591/) (2015)
- [Towards Discriminative Representation Learning for Unsupervised Person Re-identification](https://mlsanity.com/paper/2108.03439/) (2021)
- [Contrastive Pessimistic Likelihood Estimation for Semi-Supervised Classification](https://mlsanity.com/paper/1503.00269/) (2015)
- [Supervised Segmentation with Domain Adaptation for Small Sampled Orbital CT Images](https://mlsanity.com/paper/2107.00418/) (2021)

## Built on by (most cited first, 1 of 1)

- [Deep Learning in Earthquake Engineering: A Comprehensive Review](https://mlsanity.com/paper/2405.09021/) (2024)

## Abstract (arXiv)

There are numerous advantages of deep neural network surrogate modeling for response time-history prediction. However, due to the high cost of refined numerical simulations and actual experiments, the lack of data has become an unavoidable bottleneck in practical applications. An iterative self-transfer learningmethod for training neural networks based on small datasets is proposed in this study. A new mapping-based transfer learning network, named as deep adaptation network with three branches for regression (DAN-TR), is proposed. A general iterative network training strategy is developed by coupling DAN-TR and the pseudo-label strategy, and the establishment of corresponding datasets is also discussed. Finally, a complex component is selected as a case study. The results show that the proposed method can improve the model performance by near an order of magnitude on small datasets without the need of external labeled samples,well behaved pre-trainedmodels, additional artificial labeling, and complex physical/mathematical analysis.