# Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution

arXiv: https://arxiv.org/abs/2301.02068 · HTML: https://mlsanity.com/paper/2301.02068/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 51 references that are in the map.
- First posted on arXiv 5 Jan 2023.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2101.12072/) (2021)
- [Revealing the Dark Secrets of BERT](https://mlsanity.com/paper/1908.08593/) (2019)
- [Gate-Variants of Gated Recurrent Unit (GRU) Neural Networks](https://mlsanity.com/paper/1701.05923/) (2017)
- [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019)
- [Conditional Time Series Forecasting with Convolutional Neural Networks](https://mlsanity.com/paper/1703.04691/) (2017)
- [Deep AutoRegressive Networks](https://mlsanity.com/paper/1310.8499/) (2013)
- [Joint Air Quality and Weather Prediction Based on Multi-Adversarial Spatiotemporal Networks](https://mlsanity.com/paper/2012.15037/) (2020)
- [Temporal Latent Auto-Encoder: A Method for Probabilistic Multivariate Time Series Forecasting](https://mlsanity.com/paper/2101.10460/) (2021)
- [Dynamic Gaussian Mixture based Deep Generative Model For Robust Forecasting on Sparse Multivariate Time Series](https://mlsanity.com/paper/2103.02164/) (2021)

## Built on by (most cited first, 15 of 15)

- [ContiFormer: Continuous-Time Transformer for Irregular Time Series Modeling](https://mlsanity.com/paper/2402.10635/) (2024)
- [MSHyper: Multi-Scale Hypergraph Transformer for Long-Range Time Series Forecasting](https://mlsanity.com/paper/2401.09261/) (2024)
- [Joint Sensing, Communication, and AI: A Trifecta for Resilient THz User Experiences](https://mlsanity.com/paper/2305.00135/) (2023)
- [A Systematic Review for Transformer-based Long-term Series Forecasting](https://mlsanity.com/paper/2310.20218/) (2023)
- [Scalable Transformer for High Dimensional Multivariate Time Series Forecasting](https://mlsanity.com/paper/2408.04245/) (2024)
- [Tackling Data Heterogeneity in Federated Time Series Forecasting](https://mlsanity.com/paper/2411.15716/) (2024)
- [Towards Lightweight Time Series Forecasting: a Patch-wise Transformer with Weak Data Enriching](https://mlsanity.com/paper/2501.10448/) (2025)
- [UnitNorm: Rethinking Normalization for Transformers in Time Series](https://mlsanity.com/paper/2405.15903/) (2024)
- [DRFormer: Multi-Scale Transformer Utilizing Diverse Receptive Fields for Long Time-Series Forecasting](https://mlsanity.com/paper/2408.02279/) (2024)
- [Robust Multivariate Time Series Forecasting against Intra- and Inter-Series Transitional Shift](https://mlsanity.com/paper/2407.13194/) (2024)
- [Unlocking the Potential of Linear Networks for Irregular Multivariate Time Series Forecasting](https://mlsanity.com/paper/2505.00590/) (2025)
- [Accurate and Efficient Multivariate Time Series Forecasting via Offline Clustering](https://mlsanity.com/paper/2505.05738/) (2025)
- [AdaMamba: Adaptive Frequency-Gated Mamba for Long-Term Time Series Forecasting](https://mlsanity.com/paper/2604.23239/) (2026)
- [An Empirical Benchmark of Deep Time-Series Models for Smart Meter Energy Forecasting](https://mlsanity.com/paper/2608.18675/) (2026)
- [Towards Robust Time Series Learning via Capacity-Centric Modulation](https://mlsanity.com/paper/2609.39489/) (2026)

## Abstract (arXiv)

Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning. Transformer models have been adopted to deliver high prediction capacity because of the high computational self-attention mechanism. Though one could lower the complexity of Transformers by inducing the sparsity in point-wise self-attentions for LTTF, the limited information utilization prohibits the model from exploring the complex dependencies comprehensively. To this end, we propose an efficient Transformerbased model, named Conformer, which differentiates itself from existing methods for LTTF in three aspects: (i) an encoder-decoder architecture incorporating a linear complexity without sacrificing information utilization is proposed on top of sliding-window attention and Stationary and Instant Recurrent Network (SIRN); (ii) a module derived from the normalizing flow is devised to further improve the information utilization by inferring the outputs with the latent variables in SIRN directly; (iii) the inter-series correlation and temporal dynamics in time-series data are modeled explicitly to fuel the downstream self-attention mechanism. Extensive experiments on seven real-world datasets demonstrate that Conformer outperforms the state-of-the-art methods on LTTF and generates reliable prediction results with uncertainty quantification.