A Systematic Evaluation of Generated Time Series and Their Effects in Self-Supervised Pretraining
Not on the map. Why?
Abstract (arXiv)
Self-supervised Pretrained Models (PTMs) have demonstrated remarkable performance in computer vision and natural language processing tasks. These successes have prompted researchers to design PTMs for time series data. In our experiments, most self-supervised time series PTMs were surpassed by simple supervised models. We hypothesize this undesired phenomenon may be caused by data scarcity. In response, we test six time series generation methods, use the generated data in pretraining in lieu of the real data, and examine the effects on classification performance. Our results indicate that replacing a real-data pretraining set with a greater volume of only generated samples produces noticeable improvement.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 32 of its 55 references that are in the map. First posted on arXiv 15 Aug 2024.
Builds on
32 of 55 references.
References · 32
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2016Layer Normalizationcited by 5,579
- 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
- 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
- 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 606
- 2021CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputationcited by 577
- 2018Deep learning for time series classification: a reviewcited by 575
- 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
- 2021TS2Vec: Towards Universal Representation of Time Seriescited by 463
20 more
- 2020Time Series Forecasting With Deep Learning: A Surveycited by 457
- 2018The UCR Time Series Archivecited by 421
- 2019InceptionTime: Finding AlexNet for Time Series Classificationcited by 386
- 2016Time Series Classification from Scratch with Deep Neural Networks: A Strong Baselinecited by 358
- 2016Adversarial Feature Learningcited by 350
- 2018The UEA multivariate time series classification archive, 2018cited by 330
- 2022Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistencycited by 246
- 2020Time Series Data Augmentation for Deep Learning: A Surveycited by 225
- 2020An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networkscited by 128
- 2021HIVE-COTE 2.0: a new meta ensemble for time series classificationcited by 118
- 2021MultiRocket: Multiple pooling operators and transformations for fast and effective time series classificationcited by 89
- 2020Exploring Contrastive Learning in Human Activity Recognition for Healthcarecited by 73
- 2023Diffusion Models for Time Series Applications: A Surveycited by 72
- 2019TS-CHIEF: A Scalable and Accurate Forest Algorithm for Time Series Classificationcited by 67
- 2021Gated Transformer Networks for Multivariate Time Series Classificationcited by 58
- 2022Mixing Up Contrastive Learning: Self-Supervised Representation Learning for Time Seriescited by 54
- 2022Transformers in Time-series Analysis: A Tutorialcited by 45
- 2023Toward a Foundation Model for Time Series Datacited by 28
- 2019Transfer Learning for Clinical Time Series Analysis using Deep Neural Networkscited by 22
- 2022Denoising Self-attentive Sequential Recommendationcited by 20
Built on by
Citing papers per year: 2024: 1.
1 paper.