# Towards a universal neural network encoder for time series

arXiv: https://arxiv.org/abs/1805.03908 · HTML: https://mlsanity.com/paper/1805.03908/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 May 2018.

## Built on by (most cited first, 28 of 28)

- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [Deep Learning for Time Series Classification and Extrinsic Regression: A Current Survey](https://mlsanity.com/paper/2302.02515/) (2023)
- [Gated Transformer Networks for Multivariate Time Series Classification](https://mlsanity.com/paper/2103.14438/) (2021)
- [A Survey on Time-Series Pre-Trained Models](https://mlsanity.com/paper/2305.10716/) (2023)
- [Mantis: Lightweight Foundation Model for Time Series Classification](https://mlsanity.com/paper/2502.15637/) (2025)
- [dCAM: Dimension-wise Class Activation Map for Explaining Multivariate Data Series Classification](https://mlsanity.com/paper/2207.12165/) (2022)
- [MantisV2: Closing the Zero-Shot Gap in Time Series Classification with Synthetic Data and Test-Time Strategies](https://mlsanity.com/paper/2602.17868/) (2026)
- [Functional Neural Networks: Shift invariant models for functional data with applications to EEG classification](https://mlsanity.com/paper/2301.05869/) (2023)
- [Braille Letter Reading: A Benchmark for Spatio-Temporal Pattern Recognition on Neuromorphic Hardware](https://mlsanity.com/paper/2205.15864/) (2022)
- [Time-Series Classification in Smart Manufacturing Systems: An Experimental Evaluation of State-of-the-Art Machine Learning Algorithms](https://mlsanity.com/paper/2310.02812/) (2023)
- [Unlocking Pretrained Vision Transformers for Time Series Classification](https://mlsanity.com/paper/2506.08641/) (2025)
- [And what if two musical versions don't share melody, harmony, rhythm, or lyrics ?](https://mlsanity.com/paper/2210.01256/) (2022)
- [Spectral Propagation Graph Network for Few-shot Time Series Classification](https://mlsanity.com/paper/2202.04769/) (2022)
- [Sound Event Classification in an Industrial Environment: Pipe Leakage Detection Use Case](https://mlsanity.com/paper/2205.02706/) (2022)
- [Daily Physical Activity Monitoring -- Adaptive Learning from Multi-source Motion Sensor Data](https://mlsanity.com/paper/2405.16395/) (2024)
- [Capturing Temporal Components for Time Series Classification](https://mlsanity.com/paper/2406.14456/) (2024)
- [Enhancing Multivariate Time Series Classifiers through Self-Attention and Relative Positioning Infusion](https://mlsanity.com/paper/2302.06683/) (2023)
- [Control of synaptic plasticity in neural networks](https://mlsanity.com/paper/2303.07273/) (2023)
- [Enhancing Peak Assignment in 13C NMR Spectroscopy: A Novel Approach Using Multimodal Alignment](https://mlsanity.com/paper/2311.13817/) (2023)
- [Model Selection with a Shapelet-based Distance Measure for Multi-source Transfer Learning in Time Series Classification](https://mlsanity.com/paper/2409.20005/) (2024)
- [Judging a video by its bitstream cover](https://mlsanity.com/paper/2309.07361/) (2023)
- [Leveraging Compressed Frame Sizes For Ultra-Fast Video Classification](https://mlsanity.com/paper/2403.08580/) (2024)
- [Terrain Classification Enhanced with Uncertainty for Space Exploration Robots from Proprioceptive Data](https://mlsanity.com/paper/2407.03241/) (2024)
- [ECRTime: Ensemble Integration of Classification and Retrieval for Time Series Classification](https://mlsanity.com/paper/2407.14735/) (2024)
- [Real-time estimation of overt attention from dynamic features of the face using deep-learning](https://mlsanity.com/paper/2409.13084/) (2024)
- [Hardware-Friendly Delayed-Feedback Reservoir for Multivariate Time-Series Classification](https://mlsanity.com/paper/2504.11981/) (2025)
- [Adapformer: Adaptive Channel Management for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2511.14632/) (2025)
- [Seq103: A Unified Neuroevolution Framework for Compact Sequence Architecture Discovery](https://mlsanity.com/paper/2606.07664/) (2026)

## Abstract (arXiv)

We study the use of a time series encoder to learn representations that are useful on data set types with which it has not been trained on. The encoder is formed of a convolutional neural network whose temporal output is summarized by a convolutional attention mechanism. This way, we obtain a compact, fixed-length representation from longer, variable-length time series. We evaluate the performance of the proposed approach on a well-known time series classification benchmark, considering full adaptation, partial adaptation, and no adaptation of the encoder to the new data type. Results show that such strategies are competitive with the state-of-the-art, often outperforming conceptually-matching approaches. Besides accuracy scores, the facility of adaptation and the efficiency of pre-trained encoders make them an appealing option for the processing of scarcely- or non-labeled time series.