Computer Vision Self-supervised Learning Methods on Time Series
Not on the map. Why?
Abstract (arXiv)
Self-supervised learning (SSL) has had great success in both computer vision. Most of the current mainstream computer vision SSL frameworks are based on Siamese network architecture. These approaches often rely on cleverly crafted loss functions and training setups to avoid feature collapse. In this study, we evaluate if those computer-vision SSL frameworks are also effective on a different modality (textiti.e., time series). The effectiveness is experimented and evaluated on the UCR and UEA archives, and we show that the computer vision SSL frameworks can be effective even for time series. In addition, we propose a new method that improves on the recently proposed VICReg method. Our method improves on a textitcovariance term proposed in VICReg, and in addition we augment the head of the architecture by an iterative normalization layer that accelerates the convergence of the model.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 33 of its 46 references that are in the map. First posted on arXiv 2 Sep 2021.
Builds on
33 of 46 references.
References · 33
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
- 2020Exploring Simple Siamese Representation Learningcited by 2,853
- 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
- 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
21 more
- 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
- 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
- 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
- 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
- 2019ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernelscited by 336
- 2018The UEA multivariate time series classification archive, 2018cited by 330
- 2020Whitening for Self-Supervised Representation Learningcited by 278
- 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 267
- 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 223
- 2016Temporal Convolutional Networks: A Unified Approach to Action Segmentationcited by 199
- 2021On Feature Decorrelation in Self-Supervised Learningcited by 160
- 2020Deep Learning for ECG Analysis: Benchmarks and Insights from PTB-XLcited by 158
- 2020Benchmarking Multivariate Time Series Classification Algorithmscited by 145
- 2021BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representationcited by 106
- 2019Iterative Normalization: Beyond Standardization towards Efficient Whiteningcited by 92
- 2018Decorrelated Batch Normalizationcited by 83
- 2021Self-supervised representation learning from 12-lead ECG datacited by 80
- 2021Mean Shift for Self-Supervised Learningcited by 65
- 2017TimeNet: Pre-trained deep recurrent neural network for time series classificationcited by 46
- 2020Speech SIMCLR: Combining Contrastive and Reconstruction Objective for Self-supervised Speech Representation Learningcited by 33
- 2018Whitening and Coloring batch transform for GANscited by 27
Built on by
Citing papers per year: 2023: 2; 2024: 1.
3 papers.