# Mantis: Lightweight Foundation Model for Time Series Classification

arXiv: https://arxiv.org/abs/2502.15637 · HTML: https://mlsanity.com/paper/2502.15637/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 89 references that are in the map.
- First posted on arXiv 21 Feb 2025.

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline](https://mlsanity.com/paper/1611.06455/) (2016)
- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels](https://mlsanity.com/paper/1910.13051/) (2019)
- [The UEA multivariate time series classification archive, 2018](https://mlsanity.com/paper/1811.00075/) (2018)
- [xLSTM: Extended Long Short-Term Memory](https://mlsanity.com/paper/2405.04517/) (2024)

## Built on by (most cited first, 24 of 24)

- [EEG-FM-Bench: A Comprehensive Benchmark for the Systematic Evaluation and Diagnostic Analyses of EEG Foundation Models](https://mlsanity.com/paper/2508.17742/) (2025)
- [Beyond IID: How General Are Tabular Foundation Models, Really?](https://mlsanity.com/paper/2606.30410/) (2026)
- [ZARA: Training-Free Motion Time-Series Reasoning via Evidence-Grounded LLM Agents](https://mlsanity.com/paper/2508.04038/) (2025)
- [CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Data](https://mlsanity.com/paper/2508.02879/) (2025)
- [MantisV2: Closing the Zero-Shot Gap in Time Series Classification with Synthetic Data and Test-Time Strategies](https://mlsanity.com/paper/2602.17868/) (2026)
- [FeDaL: Federated Dataset Learning for General Time Series Foundation Models](https://mlsanity.com/paper/2508.04045/) (2025)
- [Pre-trained Forecasting Models: Strong Zero-Shot Feature Extractors for Time Series Classification](https://mlsanity.com/paper/2510.26777/) (2025)
- [Enhancing LLM Reasoning for Time Series Classification by Tailored Thinking and Fused Decision](https://mlsanity.com/paper/2506.00807/) (2025)
- [Leveraging Generic Time Series Foundation Models for EEG Classification](https://mlsanity.com/paper/2510.27522/) (2025)
- [Rating Quality of Diverse Time Series Data by Meta-learning from LLM Judgment](https://mlsanity.com/paper/2506.01290/) (2025)
- [Large language models surpass domain-specific architectures for antepartum electronic fetal monitoring analysis](https://mlsanity.com/paper/2509.18112/) (2025)
- [UTICA: Multi-Objective Self-Distllation Foundation Model Pretraining for Time Series Classification](https://mlsanity.com/paper/2603.01348/) (2026)
- [A foundation model for electrodermal activity data](https://mlsanity.com/paper/2603.16878/) (2026)
- [A Causal DAG Prior for Synthetic Time-Series Classification Datasets](https://mlsanity.com/paper/2606.21776/) (2026)
- [An Evaluation of Interleaved Instruction Tuning on Semantic Reasoning Performance in an Audio MLLM](https://mlsanity.com/paper/2511.02234/) (2025)
- [FMTK: A Modular Toolkit for Composable Time Series Foundation Model Pipelines](https://mlsanity.com/paper/2512.01038/) (2025)
- [EvoTSC: Evolving Feature Learning Models for Time Series Classification via Genetic Programming](https://mlsanity.com/paper/2604.25499/) (2026)
- [Modular Retrieval-Augmented Generalization for Human Action Recognition](https://mlsanity.com/paper/2605.08117/) (2026)
- [Beyond Motion Primitives: Behavioral Activity Recognition from Head-Mounted IMU](https://mlsanity.com/paper/2605.27464/) (2026)
- [Time Series as Language: A Universal Tokenizer for General-Purpose Time Series Foundation Models](https://mlsanity.com/paper/2606.09861/) (2026)
- [Self-Supervised Representation Learning: From Spectral Foundation Models to Auroral Emission Spectra](https://mlsanity.com/paper/2609.31206/) (2026)
- [When, Not How Much: Evaluating Time-Series Foundation Models on Sparse Events](https://mlsanity.com/paper/2609.39386/) (2026)
- [OntoAR-HAR: Ontology-Guided Agentic RAG with Personalized Memory for Human Activity Recognition](https://mlsanity.com/paper/2610.15292/) (2026)
- [CHEAR: Confusion-Hierarchy Evidence-Aware Reasoning for Cost-Efficient Human Activity Recognition](https://mlsanity.com/paper/2610.15298/) (2026)

## Abstract (arXiv)

While foundation models have revolutionized various domains, their application to time series classification remains rather under-explored, with existing literature predominantly focused on forecasting. To bridge this gap, we introduce Mantis, a transformer-based foundation model pre-trained exclusively on synthetic data via self-supervised contrastive learning. We demonstrate that effective tokenization is critical to unlocking the full potential of transformers, proposing a novel token generator unit. Furthermore, we introduce an enhanced test-time methodology that bridges the performance gap between Mantis and strong specialized approaches by leveraging intermediate-layer representations, self-ensembling, and cross-model embedding fusion. Extensive experiments demonstrate that Mantis establishes a new state-of-the-art, outperforming existing foundation models across four diverse dataset collections covering various application domains.