# W-Transformers : A Wavelet-based Transformer Framework for Univariate Time Series Forecasting

arXiv: https://arxiv.org/abs/2209.03945 · HTML: https://mlsanity.com/paper/2209.03945/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 89 references that are in the map.
- First posted on arXiv 8 Sep 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [Time Series Forecasting With Deep Learning: A Survey](https://mlsanity.com/paper/2004.13408/) (2020)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [Monash Time Series Forecasting Archive](https://mlsanity.com/paper/2105.06643/) (2021)
- [Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions](https://mlsanity.com/paper/1909.00590/) (2019)
- [Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks](https://mlsanity.com/paper/1802.07486/) (2018)
- [Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling](https://mlsanity.com/paper/1710.11431/) (2017)
- [Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case](https://mlsanity.com/paper/2001.08317/) (2020)
- [Darts: User-Friendly Modern Machine Learning for Time Series](https://mlsanity.com/paper/2110.03224/) (2021)
- [Probabilistic Forecasting with Temporal Convolutional Neural Network](https://mlsanity.com/paper/1906.04397/) (2019)
- [Forecasting: theory and practice](https://mlsanity.com/paper/2012.03854/) (2020)
- [Achieving Conservation of Energy in Neural Network Emulators for Climate Modeling](https://mlsanity.com/paper/1906.06622/) (2019)
- [Multi-scale Attention Flow for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2205.07493/) (2022)
- [Dynamical learning of dynamics](https://mlsanity.com/paper/1902.02875/) (2019)
- [Epicasting: An Ensemble Wavelet Neural Network (EWNet) for Forecasting Epidemics](https://mlsanity.com/paper/2206.10696/) (2022)
- [Optimized ensemble deep learning framework for scalable forecasting of dynamics containing extreme events](https://mlsanity.com/paper/2106.08968/) (2021)

## Built on by (most cited first, 12 of 12)

- [Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art](https://mlsanity.com/paper/2308.16316/) (2023)
- [When Geoscience Meets Generative AI and Large Language Models: Foundations, Trends, and Future Challenges](https://mlsanity.com/paper/2402.03349/) (2024)
- [WaveletDiff: Multilevel Wavelet Diffusion For Time Series Generation](https://mlsanity.com/paper/2510.11839/) (2025)
- [Spatiotemporal Forecasting of Traffic Flow using Wavelet-based Temporal Attention](https://mlsanity.com/paper/2407.04440/) (2024)
- [TempoKGAT: A Novel Graph Attention Network Approach for Temporal Graph Analysis](https://mlsanity.com/paper/2408.16391/) (2024)
- [Moon: A Modality Conversion-based Efficient Multivariate Time Series Anomaly Detection](https://mlsanity.com/paper/2510.01970/) (2025)
- [WaveCatBoost for Probabilistic Forecasting of Regional Air Quality Data](https://mlsanity.com/paper/2404.05482/) (2024)
- [TG-PhyNN: An Enhanced Physically-Aware Graph Neural Network framework for forecasting Spatio-Temporal Data](https://mlsanity.com/paper/2408.16379/) (2024)
- [SWIFT: Mapping Sub-series with Wavelet Decomposition Improves Time Series Forecasting](https://mlsanity.com/paper/2501.16178/) (2025)
- [Wavelet-SARIMA-Transformer: A Hybrid Model for Rainfall Forecasting](https://mlsanity.com/paper/2509.11903/) (2025)
- [A Learnable Wavelet Transformer for Long-Short Equity Trading and Risk-Adjusted Return Optimization](https://mlsanity.com/paper/2601.13435/) (2026)
- [Long-Memory Reservoir Computing for Data-Scarce Dengue Forecasting](https://mlsanity.com/paper/2607.11272/) (2026)

## Abstract (arXiv)

Deep learning utilizing transformers has recently achieved a lot of success in many vital areas such as natural language processing, computer vision, anomaly detection, and recommendation systems, among many others. Among several merits of transformers, the ability to capture long-range temporal dependencies and interactions is desirable for time series forecasting, leading to its progress in various time series applications. In this paper, we build a transformer model for non-stationary time series. The problem is challenging yet crucially important. We present a novel framework for univariate time series representation learning based on the wavelet-based transformer encoder architecture and call it W-Transformer. The proposed W-Transformers utilize a maximal overlap discrete wavelet transformation (MODWT) to the time series data and build local transformers on the decomposed datasets to vividly capture the nonstationarity and long-range nonlinear dependencies in the time series. Evaluating our framework on several publicly available benchmark time series datasets from various domains and with diverse characteristics, we demonstrate that it performs, on average, significantly better than the baseline forecasters for short-term and long-term forecasting, even for datasets that consist of only a few hundred training samples.