mlsanityData to 7 Oct 2026

SutraNets: Sub-series Autoregressive Networks for Long-Sequence, Probabilistic Forecasting

arXiv 2312.14880 ↗First posted 22 Dec 2023cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

We propose SutraNets, a novel method for neural probabilistic forecasting of long-sequence time series. SutraNets use an autoregressive generative model to factorize the likelihood of long sequences into products of conditional probabilities. When generating long sequences, most autoregressive approaches suffer from harmful error accumulation, as well as challenges in modeling long-distance dependencies. SutraNets treat long, univariate prediction as multivariate prediction over lower-frequency sub-series. Autoregression proceeds across time and across sub-series in order to ensure coherent multivariate (and, hence, high-frequency univariate) outputs. Since sub-series can be generated using fewer steps, SutraNets effectively reduce error accumulation and signal path distances. We find SutraNets to significantly improve forecasting accuracy over competitive alternatives on six real-world datasets, including when we vary the number of sub-series and scale up the depth and width of the underlying sequence models.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 50 of its 94 references that are in the map. First posted on arXiv 22 Dec 2023.

Builds on

50 of 94 references.

References · 50

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  8. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  9. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  10. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  11. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  12. 2018Datasheets for Datasetscited by 1,371
38 more
  1. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  2. 2018Model Cards for Model Reportingcited by 1,068
  3. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
  4. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  5. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
  6. 2016Pixel Recurrent Neural Networkscited by 814
  7. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  8. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 744
  9. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
  10. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 606
  11. 2013Generating Sequences With Recurrent Neural Networkscited by 549
  12. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  13. 2018Gender Bias in Coreference Resolutioncited by 438
  14. 2021Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecastingcited by 332
  15. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  16. 2015MADE: Masked Autoencoder for Distribution Estimationcited by 306
  17. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  18. 2018Efficient Neural Audio Synthesiscited by 218
  19. 2016Professor Forcing: A New Algorithm for Training Recurrent Networkscited by 214
  20. 2017A Multi-Horizon Quantile Recurrent Forecastercited by 211
  21. 2019High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processescited by 147
  22. 2020Multivariate Probabilistic Time Series Forecasting via Conditioned Normalizing Flowscited by 142
  23. 2015A Simple Way to Initialize Recurrent Networks of Rectified Linear Unitscited by 130
  24. 2017On the State of the Art of Evaluation in Neural Language Modelscited by 103
  25. 2016Hierarchical Multiscale Recurrent Neural Networkscited by 102
  26. 2016Neural Autoregressive Distribution Estimationcited by 94
  27. 2017Dilated Recurrent Neural Networkscited by 83
  28. 2014A Clockwork RNNcited by 70
  29. 2015A Hierarchical Recurrent Encoder-Decoder For Generative Context-Aware Query Suggestioncited by 68
  30. 2015Generative Image Modeling Using Spatial LSTMscited by 35
  31. 2020The Effectiveness of Discretization in Forecasting: An Empirical Study on Neural Time Series Modelscited by 18
  32. 2016Architectural Complexity Measures of Recurrent Neural Networkscited by 17
  33. 2021Probabilistic Time Series Forecasting with Implicit Quantile Networkscited by 17
  34. 2022Temporal Latent Bottleneck: Synthesis of Fast and Slow Processing Mechanisms in Sequence Learningcited by 13
  35. 2021Yformer: U-Net Inspired Transformer Architecture for Far Horizon Time Series Forecastingcited by 11
  36. 2018ARMDN: Associative and Recurrent Mixture Density Networks for eRetail Demand Forecastingcited by 9
  37. 2021MixSeq: Connecting Macroscopic Time Series Forecasting with Microscopic Time Series Datacited by 9
  38. 2018Analysing Dropout and Compounding Errors in Neural Language Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 3.

7 papers.

  1. 2024CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patternscited by 102
  2. 2024xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memoriescited by 18
  3. 2026Enhancing Multivariate Time Series Forecasting with Global Temporal Retrievalcited by 2
  4. 2025MFRS: A Multi-Frequency Reference Series Approach to Scalable and Accurate Time-Series Forecastingcited by 1
  5. 2025ATOM: A Pretrained Neural Operator for Multitask Molecular Dynamicscited by 1
  6. 2026LeapTS: Rethinking Time Series Forecasting as Adaptive Multi-Horizon Schedulingcited by 0
  7. 2026ProbRes: Volatility Learning for Probabilistic Time-Series Forecastingcited by 0