# BasisFormer: Attention-based Time Series Forecasting with Learnable and Interpretable Basis

arXiv: https://arxiv.org/abs/2310.20496 · HTML: https://mlsanity.com/paper/2310.20496/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 27 references that are in the map.
- First posted on arXiv 31 Oct 2023.

## Builds on (most cited first, 21 of 21)

- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients](https://mlsanity.com/paper/2010.07468/) (2020)
- [FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting](https://mlsanity.com/paper/2205.08897/) (2022)
- [N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting](https://mlsanity.com/paper/2201.12886/) (2022)
- [CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting](https://mlsanity.com/paper/2202.01575/) (2022)
- [Deep Learning for Audio Signal Processing](https://mlsanity.com/paper/1905.00078/) (2019)
- [DEPTS: Deep Expansion Learning for Periodic Time Series Forecasting](https://mlsanity.com/paper/2203.07681/) (2022)
- [Learning Complex Basis Functions for Invariant Representations of Audio](https://mlsanity.com/paper/1907.05982/) (2019)

## Built on by (most cited first, 30 of 41)

- [Revitalizing Multivariate Time Series Forecasting: Learnable Decomposition with Inter-Series Dependencies and Intra-Series Variations Modeling](https://mlsanity.com/paper/2402.12694/) (2024)
- [Rethinking Channel Dependence for Multivariate Time Series Forecasting: Learning from Leading Indicators](https://mlsanity.com/paper/2401.17548/) (2024)
- [Hierarchical Classification Auxiliary Network for Time Series Forecasting](https://mlsanity.com/paper/2405.18975/) (2024)
- [TimeDP: Learning to Generate Multi-Domain Time Series with Domain Prompts](https://mlsanity.com/paper/2501.05403/) (2025)
- [MCformer: Multivariate Time Series Forecasting with Mixed-Channels Transformer](https://mlsanity.com/paper/2403.09223/) (2024)
- [Deep Learning for Time Series Forecasting: A Survey](https://mlsanity.com/paper/2503.10198/) (2025)
- [The Forecast After the Forecast: A Post-Processing Shift in Time Series](https://mlsanity.com/paper/2601.20280/) (2026)
- [EMIT- Event-Based Masked Auto Encoding for Irregular Time Series](https://mlsanity.com/paper/2409.16554/) (2024)
- [Peri-midFormer: Periodic Pyramid Transformer for Time Series Analysis](https://mlsanity.com/paper/2411.04554/) (2024)
- [PGN: The RNN's New Successor is Effective for Long-Range Time Series Forecasting](https://mlsanity.com/paper/2409.17703/) (2024)
- [Interpretability for Time Series Transformers using A Concept Bottleneck Framework](https://mlsanity.com/paper/2410.06070/) (2024)
- [Learning Universal Multi-level Market Irrationality Factors to Improve Stock Return Forecasting](https://mlsanity.com/paper/2502.04737/) (2025)
- [Strada-LLM: Graph LLM for traffic prediction](https://mlsanity.com/paper/2410.20856/) (2024)
- [WaveRoRA: Wavelet Rotary Route Attention for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2410.22649/) (2024)
- [Investigating Compositional Reasoning in Time Series Foundation Models](https://mlsanity.com/paper/2502.06037/) (2025)
- [HAROOD: A Benchmark for Out-of-distribution Generalization in Sensor-based Human Activity Recognition](https://mlsanity.com/paper/2512.10807/) (2025)
- [Dominant Shuffle: A Simple Yet Powerful Data Augmentation for Time-series Prediction](https://mlsanity.com/paper/2405.16456/) (2024)
- [Energy Price Modelling: A Comparative Evaluation of four Generations of Forecasting Methods](https://mlsanity.com/paper/2411.03372/) (2024)
- [SKOLR: Structured Koopman Operator Linear RNN for Time-Series Forecasting](https://mlsanity.com/paper/2506.14113/) (2025)
- [VE: Modeling Multivariate Time Series Correlation with Variate Embedding](https://mlsanity.com/paper/2409.06169/) (2024)
- [Fourier Basis Mapping: A Time-Frequency Learning Framework for Time Series Forecasting](https://mlsanity.com/paper/2507.09445/) (2025)
- [Fremer: Lightweight and Effective Frequency Transformer for Workload Forecasting in Cloud Services](https://mlsanity.com/paper/2507.12908/) (2025)
- [VARMA-Enhanced Transformer for Time Series Forecasting](https://mlsanity.com/paper/2509.04782/) (2025)
- [DeepBooTS: Dual-Stream Residual Boosting for Drift-Resilient Time-Series Forecasting](https://mlsanity.com/paper/2511.06893/) (2025)
- [FSMLP: Modelling Channel Dependencies With Simplex Theory Based Multi-Layer Perceptions In Frequency Domain](https://mlsanity.com/paper/2412.01654/) (2024)
- [Multimodal Forecasting of Sparse Intraoperative Hypotension Events Powered by Language Model](https://mlsanity.com/paper/2505.22116/) (2025)
- [Forecast-Then-Optimize Deep Learning Methods](https://mlsanity.com/paper/2506.13036/) (2025)
- [The Power of Architecture: Deep Dive into Transformer Architectures for Long-Term Time Series Forecasting](https://mlsanity.com/paper/2507.13043/) (2025)
- [Echo Flow Networks](https://mlsanity.com/paper/2509.24122/) (2025)
- [InvDec: Inverted Decoder for Multivariate Time Series Forecasting with Separated Temporal and Variate Modeling](https://mlsanity.com/paper/2510.20302/) (2025)

## Abstract (arXiv)

Bases have become an integral part of modern deep learning-based models for time series forecasting due to their ability to act as feature extractors or future references. To be effective, a basis must be tailored to the specific set of time series data and exhibit distinct correlation with each time series within the set. However, current state-of-the-art methods are limited in their ability to satisfy both of these requirements simultaneously. To address this challenge, we propose BasisFormer, an end-to-end time series forecasting architecture that leverages learnable and interpretable bases. This architecture comprises three components: First, we acquire bases through adaptive self-supervised learning, which treats the historical and future sections of the time series as two distinct views and employs contrastive learning. Next, we design a Coef module that calculates the similarity coefficients between the time series and bases in the historical view via bidirectional cross-attention. Finally, we present a Forecast module that selects and consolidates the bases in the future view based on the similarity coefficients, resulting in accurate future predictions. Through extensive experiments on six datasets, we demonstrate that BasisFormer outperforms previous state-of-the-art methods by 11.04% and 15.78% respectively for univariate and multivariate forecasting tasks. Code is available at: urlhttps://github.com/nzl5116190/Basisformer