# MTM: A Multi-Scale Token Mixing Transformer for Irregular Multivariate Time Series Classification

arXiv: https://arxiv.org/abs/2509.17809 · HTML: https://mlsanity.com/paper/2509.17809/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 51 references that are in the map.
- First posted on arXiv 22 Sep 2025.

## Builds on (most cited first, 25 of 25)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizations](https://mlsanity.com/paper/2202.07800/) (2022)
- [The UEA multivariate time series classification archive, 2018](https://mlsanity.com/paper/1811.00075/) (2018)
- [TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods](https://mlsanity.com/paper/2403.20150/) (2024)
- [GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series](https://mlsanity.com/paper/1905.12374/) (2019)
- [DUET: Dual Clustering Enhanced Multivariate Time Series Forecasting](https://mlsanity.com/paper/2412.10859/) (2024)
- [Modeling Irregular Time Series with Continuous Recurrent Units](https://mlsanity.com/paper/2111.11344/) (2021)
- [Neural Flows: Efficient Alternative to Neural ODEs](https://mlsanity.com/paper/2110.13040/) (2021)
- [Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences](https://mlsanity.com/paper/1610.09513/) (2016)
- [Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Series](https://mlsanity.com/paper/2107.14293/) (2021)
- [Modeling Missing Data in Clinical Time Series with RNNs](https://mlsanity.com/paper/1606.04130/) (2016)
- [Time Series as Images: Vision Transformer for Irregularly Sampled Time Series](https://mlsanity.com/paper/2303.12799/) (2023)
- [Which Tokens to Use? Investigating Token Reduction in Vision Transformers](https://mlsanity.com/paper/2308.04657/) (2023)
- [Warpformer: A Multi-scale Modeling Approach for Irregular Clinical Time Series](https://mlsanity.com/paper/2306.09368/) (2023)
- [AdaCare: Explainable Clinical Health Status Representation Learning via Scale-Adaptive Feature Extraction and Recalibration](https://mlsanity.com/paper/1911.12205/) (2019)
- [RAIM: Recurrent Attentive and Intensive Model of Multimodal Patient Monitoring Data](https://mlsanity.com/paper/1807.08820/) (2018)
- [A Survey on Principles, Models and Methods for Learning from Irregularly Sampled Time Series](https://mlsanity.com/paper/2012.00168/) (2020)
- [Learning from Irregularly-Sampled Time Series: A Missing Data Perspective](https://mlsanity.com/paper/2008.07599/) (2020)
- [MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records](https://mlsanity.com/paper/2102.02340/) (2021)
- [NRTSI: Non-Recurrent Time Series Imputation](https://mlsanity.com/paper/2102.03340/) (2021)
- [MM-Path: Multi-modal, Multi-granularity Path Representation Learning -- Extended Version](https://mlsanity.com/paper/2411.18428/) (2024)
- [A Multi-Scale Decomposition MLP-Mixer for Time Series Analysis](https://mlsanity.com/paper/2310.11959/) (2023)

## Built on by (most cited first, 2 of 2)

- [CASE-NET: Deep Spatio-Temporal Representation Learning via Causal Attention and Channel Recalibration for Multivariate Time Series Classification](https://mlsanity.com/paper/2605.22043/) (2026)
- [Revitalizing Medical Time Series with Vision-Informed Retrieval: A Vision-Language Perspective](https://mlsanity.com/paper/2609.34652/) (2026)

## Abstract (arXiv)

Irregular multivariate time series (IMTS) is characterized by the lack of synchronized observations across its different channels. In this paper, we point out that this channel-wise asynchrony can lead to poor channel-wise modeling of existing deep learning methods. To overcome this limitation, we propose MTM, a multi-scale token mixing transformer for the classification of IMTS. We find that the channel-wise asynchrony can be alleviated by down-sampling the time series to coarser timescales, and propose to incorporate a masked concat pooling in MTM that gradually down-samples IMTS to enhance the channel-wise attention modules. Meanwhile, we propose a novel channel-wise token mixing mechanism which proactively chooses important tokens from one channel and mixes them with other channels, to further boost the channel-wise learning of our model. Through extensive experiments on real-world datasets and comparison with state-of-the-art methods, we demonstrate that MTM consistently achieves the best performance on all the benchmarks, with improvements of up to 3.8% in AUPRC for classification.