mlsanityData to 7 Oct 2026

User-friendly Foundation Model Adapters for Multivariate Time Series Classification

arXiv 2409.12264 ↗First posted 18 Sep 2024cs.LG stat.MLcited by 3

Not on the map. Why?

Abstract (arXiv)

Foundation models, while highly effective, are often resource-intensive, requiring substantial inference time and memory. This paper addresses the challenge of making these models more accessible with limited computational resources by exploring dimensionality reduction techniques. Our goal is to enable users to run large pre-trained foundation models on standard GPUs without sacrificing performance. We investigate classical methods such as Principal Component Analysis alongside neural network-based adapters, aiming to reduce the dimensionality of multivariate time series data while preserving key features. Our experiments show up to a 10x speedup compared to the baseline model, without performance degradation, and enable up to 4.5x more datasets to fit on a single GPU, paving the way for more user-friendly and scalable foundation models.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 24 of its 37 references that are in the map. First posted on arXiv 18 Sep 2024.

Builds on

24 of 37 references.

References · 24

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  7. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  8. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  9. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  10. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
  11. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  12. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
12 more
  1. 2019ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernelscited by 336
  2. 2018The UEA multivariate time series classification archive, 2018cited by 330
  3. 2017Soft-DTW: a Differentiable Loss Function for Time-Seriescited by 265
  4. 2020Time Series Data Augmentation for Deep Learning: A Surveycited by 225
  5. 2023TimeGPT-1cited by 149
  6. 2020An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networkscited by 128
  7. 2013A Time Series Forest for Classification and Feature Extractioncited by 69
  8. 2024SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attentioncited by 43
  9. 2024Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfercited by 31
  10. 2023NuTime: Numerically Multi-Scaled Embedding for Large-Scale Time-Series Pretrainingcited by 20
  11. 2024Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecastingcited by 7
  12. 2024Data Augmentation for Multivariate Time Series Classification: An Experimental Studycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Mantis: Lightweight Foundation Model for Time Series Classificationcited by 21
  2. 2026MantisV2: Closing the Zero-Shot Gap in Time Series Classification with Synthetic Data and Test-Time Strategiescited by 10
  3. 2026The Multiverse of Time Series Machine Learning: an Archive for Multivariate Time Series Classificationcited by 3