# VSFormer: Value and Shape-Aware Transformer with Prior-Enhanced Self-Attention for Multivariate Time Series Classification

arXiv: https://arxiv.org/abs/2412.16515 · HTML: https://mlsanity.com/paper/2412.16515/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 21 Dec 2024.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels](https://mlsanity.com/paper/1910.13051/) (2019)
- [The UEA multivariate time series classification archive, 2018](https://mlsanity.com/paper/1811.00075/) (2018)
- [Unsupervised Scalable Representation Learning for Multivariate Time Series](https://mlsanity.com/paper/1901.10738/) (2019)
- [MINIROCKET: A Very Fast (Almost) Deterministic Transform for Time Series Classification](https://mlsanity.com/paper/2012.08791/) (2020)
- [Benchmarking Multivariate Time Series Classification Algorithms](https://mlsanity.com/paper/2007.13156/) (2020)
- [Multivariate LSTM-FCNs for Time Series Classification](https://mlsanity.com/paper/1801.04503/) (2018)
- [HIVE-COTE 2.0: a new meta ensemble for time series classification](https://mlsanity.com/paper/2104.07551/) (2021)
- [Graph-Guided Network for Irregularly Sampled Multivariate Time Series](https://mlsanity.com/paper/2110.05357/) (2021)
- [A Time Series Forest for Classification and Feature Extraction](https://mlsanity.com/paper/1302.2277/) (2013)
- [Multivariate Time Series Classification with WEASEL+MUSE](https://mlsanity.com/paper/1711.11343/) (2017)
- [The Canonical Interval Forest (CIF) Classifier for Time Series Classification](https://mlsanity.com/paper/2008.09172/) (2020)
- [Fast and Accurate Time Series Classification with WEASEL](https://mlsanity.com/paper/1701.07681/) (2017)
- [Improving Position Encoding of Transformers for Multivariate Time Series Classification](https://mlsanity.com/paper/2305.16642/) (2023)
- [How to Train Your Flare Prediction Model: Revisiting Robust Sampling of Rare Events](https://mlsanity.com/paper/2103.07542/) (2021)
- [Predicting Solar Flares Using a Long Short-Term Memory Network](https://mlsanity.com/paper/1905.07095/) (2019)
- [FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification](https://mlsanity.com/paper/2302.09818/) (2023)
- [TodyNet: Temporal Dynamic Graph Neural Network for Multivariate Time Series Classification](https://mlsanity.com/paper/2304.05078/) (2023)
- [All-Clear Flare Prediction Using Interval-based Time Series Classifiers](https://mlsanity.com/paper/2105.01202/) (2021)
- [Multivariate Time Series Classification with Hierarchical Variational Graph Pooling](https://mlsanity.com/paper/2010.05649/) (2020)

## Built on by (most cited first, 2 of 2)

- [A Comparative Study on How Data Normalization Affects Zero-Shot Generalization in Time Series Foundation Models](https://mlsanity.com/paper/2512.02833/) (2025)
- [MTS-DMAE: Dual-Masked Autoencoder for Unsupervised Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2509.16078/) (2025)

## Abstract (arXiv)

Multivariate time series classification is a crucial task in data mining, attracting growing research interest due to its broad applications. While many existing methods focus on discovering discriminative patterns in time series, real-world data does not always present such patterns, and sometimes raw numerical values can also serve as discriminative features. Additionally, the recent success of Transformer models has inspired many studies. However, when applying to time series classification, the self-attention mechanisms in Transformer models could introduce classification-irrelevant features, thereby compromising accuracy. To address these challenges, we propose a novel method, VSFormer, that incorporates both discriminative patterns (shape) and numerical information (value). In addition, we extract class-specific prior information derived from supervised information to enrich the positional encoding and provide classification-oriented self-attention learning, thereby enhancing its effectiveness. Extensive experiments on all 30 UEA archived datasets demonstrate the superior performance of our method compared to SOTA models. Through ablation studies, we demonstrate the effectiveness of the improved encoding layer and the proposed self-attention mechanism. Finally, We provide a case study on a real-world time series dataset without discriminative patterns to interpret our model.