Time-Series Forecasting in Smart Manufacturing Systems: An Experimental Evaluation of the State-of-the-art Algorithms
Not on the map. Why?
Abstract (arXiv)
TSF is growing in various domains including manufacturing. Although numerous TSF algorithms have been developed recently, the validation and evaluation of algorithms hold substantial value for researchers and practitioners and are missing. This study aims to fill this gap by evaluating the SoTA TSF algorithms on thirteen manufacturing datasets, focusing on their applicability in manufacturing. Each algorithm was selected based on its TSF category to ensure a representative set of algorithms. The evaluation includes different scenarios to evaluate the models using two problem categories and two forecasting horizons. To evaluate the performance, the WAPE was calculated, and additional post hoc analyses were conducted to assess the significance of observed differences. Only algorithms with codes from open-source libraries were utilized, and no hyperparameter tuning was done. This allowed us to evaluate the algorithms as "out-of-the-box" solutions that can be easily implemented, ensuring their usability within the manufacturing by practitioners with limited technical knowledge. This aligns to facilitate the adoption of these techniques in smart manufacturing systems. Based on the results, transformer and MLP-based architectures demonstrated the best performance with MLP-based architecture winning the most scenarios. For univariate TSF, PatchTST emerged as the most robust, particularly for long-term horizons, while for multivariate problems, MLP-based architectures like N-HITS and TiDE showed superior results. The study revealed that simpler algorithms like XGBoost could outperform complex algorithms in certain tasks. These findings challenge the assumption that more sophisticated models produce better results. Additionally, the research highlighted the importance of computational resource considerations, showing variations in runtime and memory usage across different algorithms.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 35 of its 90 references that are in the map. First posted on arXiv 26 Nov 2024.
Builds on
35 of 90 references.
References · 35
- 2017Attention Is All You Needcited by 59,932
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
- 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
- 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
- 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
- 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
- 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
- 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 748
- 2018Deep learning for time series classification: a reviewcited by 575
- 2020Time Series Forecasting With Deep Learning: A Surveycited by 457
- 2018The UCR Time Series Archivecited by 421
- 2019ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernelscited by 336
23 more
- 2021SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interactioncited by 335
- 2019Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019cited by 270
- 2019Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directionscited by 172
- 2019Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecastingcited by 161
- 2018Temporal Pattern Attention for Multivariate Time Series Forecastingcited by 125
- 2020Deep Learning for Time Series Forecasting: Tutorial and Literature Surveycited by 117
- 2023Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictorscited by 110
- 2023Bake off redux: a review and experimental evaluation of recent time series classification algorithmscited by 99
- 2021Darts: User-Friendly Modern Machine Learning for Time Seriescited by 90
- 2017Dilated Recurrent Neural Networkscited by 83
- 2020Principles and Algorithms for Forecasting Groups of Time Series: Locality and Globalitycited by 70
- 2021Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with NBEATSxcited by 63
- 2018Transfer learning for time series classificationcited by 60
- 2020Machine Learning Advances for Time Series Forecastingcited by 49
- 2021An Experimental Review on Deep Learning Architectures for Time Series Forecastingcited by 47
- 2019Dataset2Vec: Learning Dataset Meta-Featurescited by 37
- 2020Forecasting: theory and practicecited by 36
- 2022Forecast combinations: an over 50-year reviewcited by 34
- 2016The Great Time Series Classification Bake Off: An Experimental Evaluation of Recently Proposed Algorithms. Extended Versioncited by 19
- 2023Time-Series Pattern Recognition in Smart Manufacturing Systems: A Literature Review and Ontologycited by 13
- 2023Methods for Quantifying Dataset Similarity: a Review, Taxonomy and Comparisoncited by 11
- 2021Parameter Efficient Deep Probabilistic Forecastingcited by 8
- 2023Time-Series Classification in Smart Manufacturing Systems: An Experimental Evaluation of State-of-the-Art Machine Learning Algorithmscited by 7
Built on by
Citing papers per year: 2025: 1.
1 paper.