# UTICA: Multi-Objective Self-Distllation Foundation Model Pretraining for Time Series Classification

arXiv: https://arxiv.org/abs/2603.01348 · HTML: https://mlsanity.com/paper/2603.01348/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 16 references that are in the map.
- First posted on arXiv 2 Mar 2026.

## Builds on (most cited first, 15 of 15)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [The UEA multivariate time series classification archive, 2018](https://mlsanity.com/paper/1811.00075/) (2018)
- [TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning](https://mlsanity.com/paper/2505.23719/) (2025)
- [Spreading vectors for similarity search](https://mlsanity.com/paper/1806.03198/) (2018)
- [Mantis: Lightweight Foundation Model for Time Series Classification](https://mlsanity.com/paper/2502.15637/) (2025)
- [MIRA: Medical Time Series Foundation Model for Real-World Health Data](https://mlsanity.com/paper/2506.07584/) (2025)
- [NuTime: Numerically Multi-Scaled Embedding for Large-Scale Time-Series Pretraining](https://mlsanity.com/paper/2310.07402/) (2023)
- [Leveraging Generic Time Series Foundation Models for EEG Classification](https://mlsanity.com/paper/2510.27522/) (2025)
- [BatteryBERT for Realistic Battery Fault Detection Using Point-Masked Signal Modeling](https://mlsanity.com/paper/2506.15712/) (2025)

## Built on by (most cited first, 1 of 1)

- [WinoTS: Wavelet-based Self-Distillation for Time Series Models](https://mlsanity.com/paper/2609.39337/) (2026)

## Abstract (arXiv)

Self-supervised foundation models have achieved remarkable success across domains, including time series. However, the potential of non-contrastive methods, a paradigm that has driven significant advances in computer vision, remains underexplored for time series. In this work, we adapt DINOv2-style self-distillation to pretrain a time series foundation model, building on the Mantis tokenizer and transformer encoder architecture as our backbone. Through a student-teacher framework, our method Utica learns representations that capture both temporal invariance via augmented crops and fine-grained local structure via patch masking. Our approach achieves state-of-the-art classification performance on both UCR and UEA benchmarks. These results suggest that non-contrastive methods are a promising and complementary pretraining strategy for time series foundation models.