# Sequence Complementor: Complementing Transformers For Time Series Forecasting with Learnable Sequences

arXiv: https://arxiv.org/abs/2501.02735 · HTML: https://mlsanity.com/paper/2501.02735/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jan 2025.

## Built on by (most cited first, 3 of 3)

- [Channel Normalization for Time Series Channel Identification](https://mlsanity.com/paper/2506.00432/) (2025)
- [EVTP-IVS: Effective Visual Token Pruning For Unifying Instruction Visual Segmentation In Multi-Modal Large Language Models](https://mlsanity.com/paper/2508.11886/) (2025)
- [Progressive Memory Transformer: Memory-Aware Attention for Time-Series](https://mlsanity.com/paper/2609.31351/) (2026)

## Abstract (arXiv)

Since its introduction, the transformer has shifted the development trajectory away from traditional models (e.g., RNN, MLP) in time series forecasting, which is attributed to its ability to capture global dependencies within temporal tokens. Follow-up studies have largely involved altering the tokenization and self-attention modules to better adapt Transformers for addressing special challenges like non-stationarity, channel-wise dependency, and variable correlation in time series. However, we found that the expressive capability of sequence representation is a key factor influencing Transformer performance in time forecasting after investigating several representative methods, where there is an almost linear relationship between sequence representation entropy and mean square error, with more diverse representations performing better. In this paper, we propose a novel attention mechanism with Sequence Complementors and prove feasible from an information theory perspective, where these learnable sequences are able to provide complementary information beyond current input to feed attention. We further enhance the Sequence Complementors via a diversification loss that is theoretically covered. The empirical evaluation of both long-term and short-term forecasting has confirmed its superiority over the recent state-of-the-art methods.