mlsanityData to 7 Oct 2026

MOMENT: A Family of Open Time-series Foundation Models

arXiv 2402.03885 ↗First posted 6 Feb 2024cs.LG cs.AIcited by 453

Not on the map. Why?

Cited by 20 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce MOMENT, a family of open-source foundation models for general-purpose time series analysis. Pre-training large models on time series data is challenging due to (1) the absence of a large and cohesive public time series repository, and (2) diverse time series characteristics which make multi-dataset training onerous. Additionally, (3) experimental benchmarks to evaluate these models, especially in scenarios with limited resources, time, and supervision, are still in their nascent stages. To address these challenges, we compile a large and diverse collection of public time series, called the Time series Pile, and systematically tackle time series-specific challenges to unlock large-scale multi-dataset pre-training. Finally, we build on recent work to design a benchmark to evaluate time series foundation models on diverse tasks and datasets in limited supervision settings. Experiments on this benchmark demonstrate the effectiveness of our pre-trained models with minimal data and task-specific fine-tuning. Finally, we present several interesting empirical observations about large pre-trained time series models. Pre-trained models (AutonLab/MOMENT-1-large) and Time Series Pile (AutonLab/Timeseries-PILE) are available on Huggingface.

Key facts. Cited by 453 papers in the map (data through 7 Oct 2026). Builds on 58 of its 88 references that are in the map. First posted on arXiv 6 Feb 2024.

Builds on

58 of 88 references.

References · 58

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  9. 2017Neural Discrete Representation Learningcited by 4,650
  10. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  11. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  12. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
38 more
  1. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  2. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  3. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  4. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  5. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  6. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  7. 2018Self-Attention with Relative Position Representationscited by 1,112
  8. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
  9. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
  10. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
  11. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 744
  12. 2023A decoder-only foundation model for time-series forecastingcited by 650
  13. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  14. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 606
  15. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  16. 2018Deep learning for time series classification: a reviewcited by 575
  17. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  18. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  19. 2022Transformers in Time Series: A Surveycited by 489
  20. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 463
  21. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
  22. 2018Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholdingcited by 407
  23. 2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 387
  24. 2016Time Series Classification from Scratch with Deep Neural Networks: A Strong Baselinecited by 358
  25. 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 342
  26. 2022Scaling Language-Image Pre-training via Maskingcited by 317
  27. 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 267
  28. 2021Monash Time Series Forecasting Archivecited by 256
  29. 2018Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applicationscited by 223
  30. 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 223
  31. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 198
  32. 2023TimeGPT-1cited by 149
  33. 2020Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progresscited by 148
  34. 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 120
  35. 2023SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modelingcited by 112
  36. 2022Measuring the Carbon Intensity of AI in Cloud Instancescited by 110
  37. 2016Multi-Scale Convolutional Neural Networks for Time Series Classificationcited by 99
  38. 2020Meta-learning framework with applications to zero-shot time-series forecastingcited by 70

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 66; 2025: 185; 2026: 202.

453 papers.

Sort:

By area:

  1. 2024Chronos: Learning the Language of Time Seriescited by 712
  2. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 280
  3. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 191
  4. 2025Sundial: A Family of Highly Capable Time Series Foundation Modelscited by 168
  5. 2024ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilitiescited by 104
  6. 2024UniTS: A Unified Multi-Task Time Series Modelcited by 93
  7. 2024ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Datacited by 92
  8. 2024Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Expertscited by 88
  9. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 87
  10. 2024Timer-XL: Long-Context Transformers for Unified Time Series Forecastingcited by 83
  11. 2024PaPaGei: Open Foundation Models for Optical Physiological Signalscited by 64
  12. 2025Moirai 2.0: When Less Is More for Time Series Forecastingcited by 59
38 more
  1. 2025TAB: Unified Benchmarking of Time Series Anomaly Detection Methodscited by 58
  2. 2024Scaling Wearable Foundation Modelscited by 57
  3. 2025This Time is Different: An Observability Perspective on Time Series Foundation Modelscited by 57
  4. 2025MTBench: A Multimodal Time Series Benchmark for Temporal Reasoning and Question Answeringcited by 50
  5. 2024TimeSeriesExam: A time series understanding examcited by 47
  6. 2025DBLoss: Decomposition-based Loss Function for Time Series Forecastingcited by 44
  7. 2024Retrieval Augmented Time Series Forecastingcited by 40
  8. 2024Towards a General Time Series Anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoderscited by 37
  9. 2025TS-RAG: Retrieval-Augmented Generation based Time Series Foundation Models are Stronger Zero-Shot Forecastercited by 36
  10. 2025Pulse-PPG: An Open-Source Field-Trained PPG Foundation Model for Wearable Applications Across Lab and Field Settingscited by 35
  11. 2025ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with Large-Scale Multitask Datasetcited by 35
  12. 2025Time Series Forecasting via Reasoning: A Slow-Thinking Approach with Reinforcement Fine-Tuned LLMscited by 34
  13. 2024Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfercited by 31
  14. 2024In-Context Fine-Tuning for Time-Series Foundation Modelscited by 31
  15. 2024Towards Neural Scaling Laws for Time Series Foundation Modelscited by 30
  16. 2025EEG Foundation Models: A Critical Review of Current Progress and Future Directionscited by 30
  17. 2024TS-Reasoner: Domain-Oriented Time Series Inference Agents for Reasoning and Automated Analysiscited by 28
  18. 2025LightGTS: A Lightweight General Time Series Forecasting Modelcited by 28
  19. 2025TimeKAN: KAN-based Frequency Decomposition Learning Architecture for Long-term Time Series Forecastingcited by 27
  20. 2024BrainWave: A Brain Signal Foundation Model for Clinical Applicationscited by 26
  21. 2024Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecastingcited by 26
  22. 2024Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenizationcited by 26
  23. 2025Harnessing Vision Models for Time Series Analysis: A Surveycited by 26
  24. 2024Zero-shot forecasting of chaotic systemscited by 25
  25. 2026Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scalingcited by 24
  26. 2024Empowering Time Series Analysis with Foundation Models: A Comprehensive Surveycited by 23
  27. 2024Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shiftcited by 23
  28. 2024Specialized Foundation Models Struggle to Beat Supervised Baselinescited by 23
  29. 2025Mantis: Lightweight Foundation Model for Time Series Classificationcited by 21
  30. 2025MIRA: Medical Time Series Foundation Model for Real-World Health Datacited by 21
  31. 2025EEG-FM-Bench: A Comprehensive Benchmark for the Systematic Evaluation and Diagnostic Analyses of EEG Foundation Modelscited by 21
  32. 2024Exploring Representations and Interventions in Time Series Foundation Modelscited by 19
  33. 2025Foundation Models for Time Series: A Surveycited by 19
  34. 2025Can Slow-thinking LLMs Reason Over Time? Empirical Studies in Time Series Forecastingcited by 19
  35. 2024Are Self-Attentions Effective for Time Series Forecasting?cited by 18
  36. 2025FreEformer: Frequency Enhanced Transformer for Multivariate Time Series Forecastingcited by 18
  37. 2024RelCon: Relative Contrastive Learning for a Motion Foundation Model for Wearable Datacited by 17
  38. 2024Scaling-laws for Large Time-series Modelscited by 16

50 most cited of 453 All 453 (JSON)