mlsanityData to 14 Oct 2026

MTS-DMAE: Dual-Masked Autoencoder for Unsupervised Multivariate Time Series Representation Learning

arXiv 2509.16078 ↗First posted 19 Sep 2025cs.LG cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Unsupervised multivariate time series (MTS) representation learning aims to extract compact and informative representations from raw sequences without relying on labels, enabling efficient transfer to diverse downstream tasks. In this paper, we propose Dual-Masked Autoencoder (DMAE), a novel masked time-series modeling framework for unsupervised MTS representation learning. DMAE formulates two complementary pretext tasks: (1) reconstructing masked values based on visible attributes, and (2) estimating latent representations of masked features, guided by a teacher encoder. To further improve representation quality, we introduce a feature-level alignment constraint that encourages the predicted latent representations to align with the teacher's outputs. By jointly optimizing these objectives, DMAE learns temporally coherent and semantically rich representations. Comprehensive evaluations across classification, regression, and forecasting tasks demonstrate that our approach achieves consistent and superior performance over competitive baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 51 references that are in the map. First posted on arXiv 19 Sep 2025.

Builds on

36 of 51 references.

References · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  7. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  8. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  9. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  10. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  11. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  12. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
24 more
  1. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  2. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 471
  3. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 453
  4. 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 345
  5. 2019ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernelscited by 339
  6. 2018The UEA multivariate time series classification archive, 2018cited by 338
  7. 2022Scaling Language-Image Pre-training via Maskingcited by 317
  8. 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 269
  9. 2022Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistencycited by 250
  10. 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 224
  11. 2022CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecastingcited by 144
  12. 2023SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modelingcited by 115
  13. 2022Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusioncited by 96
  14. 2023Ti-MAE: Self-Supervised Masked Time Series Autoencoderscited by 63
  15. 2018SOM-VAE: Interpretable Discrete Representation Learning on Time Seriescited by 60
  16. 2023Time Series Contrastive Learning with Information-Aware Augmentationscited by 52
  17. 2017TimeNet: Pre-trained deep recurrent neural network for time series classificationcited by 46
  18. 2023FOCAL: Contrastive Learning for Multimodal Time-Series Sensing Signals in Factorized Orthogonal Latent Spacecited by 33
  19. 2018Improving Clinical Predictions through Unsupervised Time Series Representation Learningcited by 24
  20. 2022Decoupling Local and Global Representations of Time Seriescited by 17
  21. 2020Monash University, UEA, UCR Time Series Extrinsic Regression Archivecited by 10
  22. 2023Simple Contrastive Representation Learning for Time Series Forecastingcited by 7
  23. 2024VSFormer: Value and Shape-Aware Transformer with Prior-Enhanced Self-Attention for Multivariate Time Series Classificationcited by 2
  24. 2022DBT-DMAE: An Effective Multivariate Time Series Pre-Train Model under Missing Datacited by 1

Built on by

Not cited yet.