mlsanityData to 14 Oct 2026

Optimizing Time Series Forecasting Architectures: A Hierarchical Neural Architecture Search Approach

arXiv 2406.05088 ↗First posted 7 Jun 2024cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

The rapid development of time series forecasting research has brought many deep learning-based modules in this field. However, despite the increasing amount of new forecasting architectures, it is still unclear if we have leveraged the full potential of these existing modules within a properly designed architecture. In this work, we propose a novel hierarchical neural architecture search approach for time series forecasting tasks. With the design of a hierarchical search space, we incorporate many architecture types designed for forecasting tasks and allow for the efficient combination of different forecasting architecture modules. Results on long-term-time-series-forecasting tasks show that our approach can search for lightweight high-performing forecasting architectures across different forecasting tasks.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 65 of its 90 references that are in the map. First posted on arXiv 7 Jun 2024.

Builds on

65 of 90 references.

References · 65

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  9. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  10. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  11. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  12. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
38 more
  1. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  2. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  3. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  4. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  5. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  7. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  8. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  9. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  10. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  11. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  12. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  13. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,013
  14. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 931
  15. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 879
  16. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  17. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  18. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 761
  19. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 753
  20. 2023A decoder-only foundation model for time-series forecastingcited by 692
  21. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 607
  22. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 479
  23. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
  24. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
  25. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 344
  26. 2021Monash Time Series Forecasting Archivecited by 267
  27. 2020Neural Architecture Search without Trainingcited by 244
  28. 2023TSMixer: An All-MLP Architecture for Time Series Forecastingcited by 240
  29. 2017A Multi-Horizon Quantile Recurrent Forecastercited by 211
  30. 2021Zero-Cost Proxies for Lightweight NAScited by 199
  31. 2017Hierarchical Representations for Efficient Architecture Searchcited by 191
  32. 2021AutoFormer: Searching Transformers for Visual Recognitioncited by 188
  33. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 179
  34. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  35. 2019Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directionscited by 172
  36. 2018Auto-Keras: An Efficient Neural Architecture Search Systemcited by 171
  37. 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 131
  38. 2021Rethinking Architecture Selection in Differentiable NAScited by 92

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Neural Architecture Search for global multi-step Forecasting of Energy Production Time Seriescited by 0
  2. 2025MSTN: A Lightweight and Fast Model for General TimeSeries Analysiscited by 0
  3. 2026Neural Architecture Search for Traffic Prediction: A Survey of Methods, Challenges, and Future Directionscited by 0