mlsanityData to 14 Oct 2026

Mantis: Lightweight Foundation Model for Time Series Classification

arXiv 2502.15637 ↗First posted 21 Feb 2025cs.LG cs.AI stat.MLcited by 24

Not on the map. Why?

Abstract (arXiv)

While foundation models have revolutionized various domains, their application to time series classification remains rather under-explored, with existing literature predominantly focused on forecasting. To bridge this gap, we introduce Mantis, a transformer-based foundation model pre-trained exclusively on synthetic data via self-supervised contrastive learning. We demonstrate that effective tokenization is critical to unlocking the full potential of transformers, proposing a novel token generator unit. Furthermore, we introduce an enhanced test-time methodology that bridges the performance gap between Mantis and strong specialized approaches by leveraging intermediate-layer representations, self-ensembling, and cross-model embedding fusion. Extensive experiments demonstrate that Mantis establishes a new state-of-the-art, outperforming existing foundation models across four diverse dataset collections covering various application domains.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 49 of its 89 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

49 of 89 references.

References · 49

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2023GPT-4 Technical Reportcited by 18,396
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  8. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  9. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  10. 2016Layer Normalizationcited by 5,669
  11. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
37 more
  1. 2022A ConvNet for the 2020scited by 3,579
  2. 2017Decoupled Weight Decay Regularizationcited by 1,831
  3. 2020GLU Variants Improve Transformercited by 1,804
  4. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  5. 2019Root Mean Square Layer Normalizationcited by 1,480
  6. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  7. 2023A decoder-only foundation model for time-series forecastingcited by 692
  8. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  9. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  10. 2018Deep learning for time series classification: a reviewcited by 576
  11. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 482
  12. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 471
  13. 2018The UCR Time Series Archivecited by 431
  14. 2016Time Series Classification from Scratch with Deep Neural Networks: A Strong Baselinecited by 360
  15. 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 345
  16. 2019ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernelscited by 339
  17. 2018The UEA multivariate time series classification archive, 2018cited by 338
  18. 2024xLSTM: Extended Long Short-Term Memorycited by 327
  19. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
  20. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 289
  21. 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 269
  22. 2022Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistencycited by 250
  23. 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 224
  24. 2024CBraMod: A Criss-Cross Brain Foundation Model for EEG Decodingcited by 199
  25. 2020Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progresscited by 151
  26. 2024UniTS: A Unified Multi-Task Time Series Modelcited by 94
  27. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 88
  28. 2022Data augmentation for learning predictive models on EEG: a systematic comparisoncited by 33
  29. 2022WOODS: Benchmarks for Out-of-Distribution Generalization in Time Seriescited by 30
  30. 2018Towards a universal neural network encoder for time seriescited by 28
  31. 2023NuTime: Numerically Multi-Scaled Embedding for Large-Scale Time-Series Pretrainingcited by 20
  32. 2024A Closer Look at Benchmarking Self-Supervised Pre-training with Image Classificationcited by 19
  33. 2024Mamba4Cast: Efficient Zero-Shot Time Series Forecasting with State Space Modelscited by 17
  34. 2025AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecastingcited by 15
  35. 2020EMOPAIN Challenge 2020: Multimodal Pain Evaluation from Facial and Bodily Expressionscited by 11
  36. 2022Fast and Robust Video-Based Exercise Classification via Body Pose Tracking and Scalable Multivariate Time Series Classifierscited by 8
  37. 2024User-friendly Foundation Model Adapters for Multivariate Time Series Classificationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 13.

24 papers.

Sort:

By area:

  1. 2025EEG-FM-Bench: A Comprehensive Benchmark for the Systematic Evaluation and Diagnostic Analyses of EEG Foundation Modelscited by 23
  2. 2026Beyond IID: How General Are Tabular Foundation Models, Really?cited by 20
  3. 2025ZARA: Training-Free Motion Time-Series Reasoning via Evidence-Grounded LLM Agentscited by 15
  4. 2025CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Datacited by 13
  5. 2026MantisV2: Closing the Zero-Shot Gap in Time Series Classification with Synthetic Data and Test-Time Strategiescited by 11
  6. 2025FeDaL: Federated Dataset Learning for General Time Series Foundation Modelscited by 10
  7. 2025Pre-trained Forecasting Models: Strong Zero-Shot Feature Extractors for Time Series Classificationcited by 9
  8. 2025Enhancing LLM Reasoning for Time Series Classification by Tailored Thinking and Fused Decisioncited by 8
  9. 2025Leveraging Generic Time Series Foundation Models for EEG Classificationcited by 3
  10. 2025Rating Quality of Diverse Time Series Data by Meta-learning from LLM Judgmentcited by 1
  11. 2025Large language models surpass domain-specific architectures for antepartum electronic fetal monitoring analysiscited by 1
  12. 2026UTICA: Multi-Objective Self-Distllation Foundation Model Pretraining for Time Series Classificationcited by 1
12 more
  1. 2026A foundation model for electrodermal activity datacited by 1
  2. 2026A Causal DAG Prior for Synthetic Time-Series Classification Datasetscited by 1
  3. 2025An Evaluation of Interleaved Instruction Tuning on Semantic Reasoning Performance in an Audio MLLMcited by 0
  4. 2025FMTK: A Modular Toolkit for Composable Time Series Foundation Model Pipelinescited by 0
  5. 2026EvoTSC: Evolving Feature Learning Models for Time Series Classification via Genetic Programmingcited by 0
  6. 2026Modular Retrieval-Augmented Generalization for Human Action Recognitioncited by 0
  7. 2026Beyond Motion Primitives: Behavioral Activity Recognition from Head-Mounted IMUcited by 0
  8. 2026Time Series as Language: A Universal Tokenizer for General-Purpose Time Series Foundation Modelscited by 0
  9. 2026Self-Supervised Representation Learning: From Spectral Foundation Models to Auroral Emission Spectracited by 0
  10. 2026When, Not How Much: Evaluating Time-Series Foundation Models on Sparse Eventscited by 0
  11. 2026OntoAR-HAR: Ontology-Guided Agentic RAG with Personalized Memory for Human Activity Recognitioncited by 0
  12. 2026CHEAR: Confusion-Hierarchy Evidence-Aware Reasoning for Cost-Efficient Human Activity Recognitioncited by 0