Mitigating Data Scarcity in Time Series Analysis: A Foundation Model with Series-Symbol Data Generation
Not on the map. Why?
Abstract (arXiv)
Foundation models for time series analysis (TSA) have attracted significant attention. However, challenges such as data scarcity and data imbalance continue to hinder their development. To address this, we consider modeling complex systems through symbolic expressions that serve as semantic descriptors of time series. Building on this concept, we introduce a series-symbol (S2) dual-modulity data generation mechanism, enabling the unrestricted creation of high-quality time series data paired with corresponding symbolic representations. Leveraging the S2 dataset, we develop SymTime, a pre-trained foundation model for TSA. SymTime demonstrates competitive performance across five major TSA tasks when fine-tuned with downstream task, rivaling foundation models pre-trained on real-world datasets. This approach underscores the potential of dual-modality data generation and pretraining mechanisms in overcoming data scarcity and enhancing task performance.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 74 of its 114 references that are in the map. First posted on arXiv 21 Feb 2025.
Builds on
74 of 114 references.
References · 74
- 2017Attention Is All You Needcited by 59,932
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
- 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
- 2020Reformer: The Efficient Transformercited by 1,658
- 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
38 more
- 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
- 2020Knowledge Distillation: A Surveycited by 1,578
- 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
- 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
- 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
- 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
- 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
- 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
- 2023A decoder-only foundation model for time-series forecastingcited by 650
- 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
- 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
- 2018Deep learning for time series classification: a reviewcited by 575
- 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 536
- 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
- 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
- 2018Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholdingcited by 407
- 2019InceptionTime: Finding AlexNet for Time Series Classificationcited by 386
- 2022Non-stationary Transformers: Exploring the Stationarity in Time Series Forecastingcited by 385
- 2021Explaining Neural Scaling Lawscited by 353
- 2021Transformers Can Do Bayesian Inferencecited by 338
- 2019ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernelscited by 336
- 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 280
- 2019Deep Learning for Symbolic Mathematicscited by 266
- 2021Monash Time Series Forecasting Archivecited by 256
- 2024TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variablescited by 241
- 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
- 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 198
- 2022End-to-end symbolic regression with transformerscited by 195
- 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 191
- 2022Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structurescited by 158
- 2022ETSformer: Exponential Smoothing Transformers for Time-series Forecastingcited by 146
- 2023Scaling Laws for Generative Mixed-Modal Language Modelscited by 131
- 2024TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysiscited by 113
- 2023SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modelingcited by 112
- 2024SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameterscited by 103
- 2023ForecastPFN: Synthetically-Trained Zero-Shot Forecastingcited by 102
- 2024UniTS: A Unified Multi-Task Time Series Modelcited by 93
- 2024ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Datacited by 92
50 most cited of 74 All 74 (JSON)
Built on by
Citing papers per year: 2025: 1.
1 paper.