# ConvTimeNet: A Pre-trained Deep Convolutional Neural Network for Time Series Classification

arXiv: https://arxiv.org/abs/1904.12546 · HTML: https://mlsanity.com/paper/1904.12546/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2019.

## Built on by (most cited first, 20 of 20)

- [Voice2Series: Reprogramming Acoustic Models for Time Series Classification](https://mlsanity.com/paper/2106.09296/) (2021)
- [Explainable Artificial Intelligence (XAI) on TimeSeries Data: A Survey](https://mlsanity.com/paper/2104.00950/) (2021)
- [Deep Learning for Time Series Classification and Extrinsic Regression: A Current Survey](https://mlsanity.com/paper/2302.02515/) (2023)
- [A Survey on Time-Series Pre-Trained Models](https://mlsanity.com/paper/2305.10716/) (2023)
- [Interpretation of Time-Series Deep Models: A Survey](https://mlsanity.com/paper/2305.14582/) (2023)
- [EEG functional connectivity and deep learning for automatic diagnosis of brain disorders: Alzheimer's disease and schizophrenia](https://mlsanity.com/paper/2110.06140/) (2021)
- [Intra-domain and cross-domain transfer learning for time series data -- How transferable are the features?](https://mlsanity.com/paper/2201.04449/) (2022)
- [Transfer learning for time series classification using synthetic data generation](https://mlsanity.com/paper/2207.07897/) (2022)
- [Similarity Learning based Few Shot Learning for ECG Time Series Classification](https://mlsanity.com/paper/2202.00612/) (2022)
- [Spectral Propagation Graph Network for Few-shot Time Series Classification](https://mlsanity.com/paper/2202.04769/) (2022)
- [XAI for time-series classification leveraging image highlight methods](https://mlsanity.com/paper/2311.17110/) (2023)
- [United We Pretrain, Divided We Fail! Representation Learning for Time Series by Pretraining on 75 Datasets at Once](https://mlsanity.com/paper/2402.15404/) (2024)
- [On the Intersection of Signal Processing and Machine Learning: A Use Case-Driven Analysis Approach](https://mlsanity.com/paper/2403.17181/) (2024)
- [ECATS: Explainable-by-design concept-based anomaly detection for time series](https://mlsanity.com/paper/2405.10608/) (2024)
- [Two ways towards combining Sequential Neural Network and Statistical Methods to Improve the Prediction of Time Series](https://mlsanity.com/paper/2110.00082/) (2021)
- [Deep Neural Networks on EEG Signals to Predict Auditory Attention Score Using Gramian Angular Difference Field](https://mlsanity.com/paper/2110.12503/) (2021)
- [TCGAN: Convolutional Generative Adversarial Network for Time Series Classification and Clustering](https://mlsanity.com/paper/2309.04732/) (2023)
- [A Spatiotemporal Deep Neural Network for Fine-Grained Multi-Horizon Wind Prediction](https://mlsanity.com/paper/2309.04733/) (2023)
- [Efficient Observation Time Window Segmentation for Administrative Data Machine Learning](https://mlsanity.com/paper/2401.16537/) (2024)
- [Towards Accurate and Interpretable Time-series Forecasting: A Polynomial Learning Approach](https://mlsanity.com/paper/2603.02906/) (2026)

## Abstract (arXiv)

Training deep neural networks often requires careful hyper-parameter tuning and significant computational resources. In this paper, we propose ConvTimeNet (CTN): an off-the-shelf deep convolutional neural network (CNN) trained on diverse univariate time series classification (TSC) source tasks. Once trained, CTN can be easily adapted to new TSC target tasks via a small amount of fine-tuning using labeled instances from the target tasks. We note that the length of convolutional filters is a key aspect when building a pre-trained model that can generalize to time series of different lengths across datasets. To achieve this, we incorporate filters of multiple lengths in all convolutional layers of CTN to capture temporal features at multiple time scales. We consider all 65 datasets with time series of lengths up to 512 points from the UCR TSC Benchmark for training and testing transferability of CTN: We train CTN on a randomly chosen subset of 24 datasets using a multi-head approach with a different softmax layer for each training dataset, and study generalizability and transferability of the learned filters on the remaining 41 TSC datasets. We observe significant gains in classification accuracy as well as computational efficiency when using pre-trained CTN as a starting point for subsequent task-specific fine-tuning compared to existing state-of-the-art TSC approaches. We also provide qualitative insights into the working of CTN by: i) analyzing the activations and filters of first convolution layer suggesting the filters in CTN are generically useful, ii) analyzing the impact of the design decision to incorporate multiple length decisions, and iii) finding regions of time series that affect the final classification decision via occlusion sensitivity analysis.