mlsanityData to 7 Oct 2026

Respecting Time Series Properties Makes Deep Time Series Forecasting Perfect

arXiv 2207.10941 ↗First posted 22 Jul 2022cs.LG cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

How to handle time features shall be the core question of any time series forecasting model. Ironically, it is often ignored or misunderstood by deep-learning based models, even those baselines which are state-of-the-art. This behavior makes their inefficient, untenable and unstable. In this paper, we rigorously analyze three prevalent but deficient/unfounded deep time series forecasting mechanisms or methods from the view of time series properties, including normalization methods, multivariate forecasting and input sequence length. Corresponding corollaries and solutions are given on both empirical and theoretical basis. We thereby propose a novel time series forecasting network, i.e. RTNet, on the basis of aforementioned analysis. It is general enough to be combined with both supervised and self-supervised forecasting format. Thanks to the core idea of respecting time series properties, no matter in which forecasting format, RTNet shows obviously superior forecasting performances compared with dozens of other SOTA time series forecasting baselines in three real-world benchmark datasets. By and large, it even occupies less time complexity and memory usage while acquiring better forecasting accuracy. The source code is available at https://github.com/OrigamiSL/RTNet.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 51 of its 85 references that are in the map. First posted on arXiv 22 Jul 2022.

Builds on

51 of 85 references.

References · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  8. 2016Layer Normalizationcited by 5,579
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  10. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  11. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  12. 2020Longformer: The Long-Document Transformercited by 3,362
38 more
  1. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  2. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  3. 2020Reformer: The Efficient Transformercited by 1,658
  4. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  5. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  6. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  7. 2017Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecastingcited by 1,125
  8. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
  9. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
  10. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 748
  11. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 744
  12. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 606
  13. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
  14. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
  15. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
  16. 2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 387
  17. 2019CSPNet: A New Backbone that can Enhance Learning Capability of CNNcited by 381
  18. 2021ResNet strikes back: An improved training procedure in timmcited by 365
  19. 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 342
  20. 2021Spectral Temporal Graph Neural Network for Multivariate Time-series Forecastingcited by 298
  21. 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 267
  22. 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 223
  23. 2020Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecastingcited by 212
  24. 2017A Multi-Horizon Quantile Recurrent Forecastercited by 211
  25. 2022N-HiTS: Neural Hierarchical Interpolation for Time Series Forecastingcited by 146
  26. 2022ETSformer: Exponential Smoothing Transformers for Time-series Forecastingcited by 146
  27. 2022CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecastingcited by 143
  28. 2019Attention Interpretability Across NLP Taskscited by 71
  29. 2019Sparse Graph Attention Networkscited by 41
  30. 2021TCN: Table Convolutional Network for Web Table Interpretationcited by 24
  31. 2014Multi-Step-Ahead Time Series Prediction using Multiple-Output Support Vector Regressioncited by 17
  32. 2020Exploring the Generalizability of Spatio-Temporal Traffic Prediction: Meta-Modeling and an Analytic Frameworkcited by 12
  33. 2018Multivariate Bayesian Structural Time Series Modelcited by 11
  34. 2021Yformer: U-Net Inspired Transformer Architecture for Far Horizon Time Series Forecastingcited by 11
  35. 2020STR: Seasonal-Trend Decomposition Using Regressioncited by 9
  36. 2021Empirical Quantitative Analysis of COVID-19 Forecasting Modelscited by 8
  37. 2019PAN: Path Integral Based Convolution for Deep Graph Neural Networkscited by 5
  38. 2020Res-CR-Net, a residual network with a novel architecture optimized for the semantic segmentation of microscopy imagescited by 3

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 3.

3 papers.