Survey and Taxonomy: The Role of Data-Centric AI in Transformer-Based Time Series Forecasting
Not on the map. Why?
Abstract (arXiv)
Alongside the continuous process of improving AI performance through the development of more sophisticated models, researchers have also focused their attention to the emerging concept of data-centric AI, which emphasizes the important role of data in a systematic machine learning training process. Nonetheless, the development of models has also continued apace. One result of this progress is the development of the Transformer Architecture, which possesses a high level of capability in multiple domains such as Natural Language Processing (NLP), Computer Vision (CV) and Time Series Forecasting (TSF). Its performance is, however, heavily dependent on input data preprocessing and output data evaluation, justifying a data-centric approach to future research. We argue that data-centric AI is essential for training AI models, particularly for transformer-based TSF models efficiently. However, there is a gap regarding the integration of transformer-based TSF and data-centric AI. This survey aims to pin down this gap via the extensive literature review based on the proposed taxonomy. We review the previous research works from a data-centric AI perspective and we intend to lay the foundation work for the future development of transformer-based architecture and data-centric AI.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 53 of its 88 references that are in the map. First posted on arXiv 29 Jul 2024.
Builds on
53 of 88 references.
References · 53
- 2017Attention Is All You Needcited by 59,932
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
- 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
- 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
- 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
- 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
- 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
- 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
- 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 748
- 2020A Survey on Visual Transformercited by 653
- 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
- 2022Transformers in Time Series: A Surveycited by 489
38 more
- 2020Time Series Forecasting With Deep Learning: A Surveycited by 457
- 2022Non-stationary Transformers: Exploring the Stationarity in Time Series Forecastingcited by 385
- 2021Monash Time Series Forecasting Archivecited by 256
- 2020Time Series Data Augmentation for Deep Learning: A Surveycited by 225
- 2023Data-centric Artificial Intelligence: A Surveycited by 202
- 2022PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecastingcited by 170
- 2020Deep Learning for Time Series Forecasting: Tutorial and Literature Surveycited by 117
- 2021Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspectivecited by 116
- 2024Large Language Models for Time Series: A Surveycited by 113
- 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
- 2020Data Movement Is All You Need: A Case Study on Optimizing Transformerscited by 108
- 2021Position Information in Transformers: An Overviewcited by 103
- 2023Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecastingcited by 102
- 2021Darts: User-Friendly Modern Machine Learning for Time Seriescited by 90
- 2023A Comprehensive Survey on Applications of Transformers for Deep Learning Taskscited by 70
- 2022Eco2AI: carbon emissions tracking of machine learning models as the first step towards sustainable AIcited by 58
- 2022Scaleformer: Iterative Multi-scale Refining Transformers for Time Series Forecastingcited by 54
- 2021An Experimental Review on Deep Learning Architectures for Time Series Forecastingcited by 47
- 2021Do We Really Need Deep Learning Models for Time Series Forecasting?cited by 45
- 2022Transformers in Time-series Analysis: A Tutorialcited by 45
- 2023Improving Position Encoding of Transformers for Multivariate Time Series Classificationcited by 44
- 2023Uncertainty in Natural Language Processing: Sources, Quantification, and Applicationscited by 44
- 2023Data-centric AI: Perspectives and Challengescited by 43
- 2022TACTiS: Transformer-Attentional Copulas for Time Seriescited by 38
- 2022Data-Centric Artificial Intelligencecited by 31
- 2017Feature-based time-series analysiscited by 28
- 2021What can Data-Centric AI Learn from Data and ML Engineering?cited by 24
- 2022Preformer: Predictive Transformer with Multi-Scale Segment-wise Correlations for Long-Term Time Series Forecastingcited by 23
- 2023TransFace++: Rethinking the Face Recognition Paradigm with a Focus on Accuracy, Efficiency, and Securitycited by 22
- 2023Data-centric Graph Learning: A Surveycited by 22
- 2022First De-Trend then Attend: Rethinking Attention for Time-Series Forecastingcited by 19
- 2022Optimal Ratio for Data Splittingcited by 17
- 2022Gluformer: Transformer-Based Personalized Glucose Forecasting with Uncertainty Quantificationcited by 17
- 2021TCCT: Tightly-Coupled Convolutional Transformer on Time Series Forecastingcited by 8
- 2022Data-Centric Epidemic Forecasting: A Surveycited by 8
- 2023Client: Cross-variable Linear Integrated Enhanced Transformer for Multivariate Long-Term Time Series Forecastingcited by 8
- 2022Probabilistic Decomposition Transformer for Time Series Forecastingcited by 6
- 2023GCformer: An Efficient Framework for Accurate and Scalable Long-Term Multivariate Time Series Forecastingcited by 5
50 most cited of 53 All 53 (JSON)
Built on by
Citing papers per year: 2024: 1; 2025: 2.
3 papers.