# KEDformer:Knowledge Extraction Seasonal Trend Decomposition for Long-term Sequence Prediction

arXiv: https://arxiv.org/abs/2412.05421 · HTML: https://mlsanity.com/paper/2412.05421/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 58 references that are in the map.
- First posted on arXiv 6 Dec 2024.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [FNet: Mixing Tokens with Fourier Transforms](https://mlsanity.com/paper/2105.03824/) (2021)
- [Synthesizer: Rethinking Self-Attention in Transformer Models](https://mlsanity.com/paper/2005.00743/) (2020)
- [ETSformer: Exponential Smoothing Transformers for Time-series Forecasting](https://mlsanity.com/paper/2202.01381/) (2022)
- [Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case](https://mlsanity.com/paper/2001.08317/) (2020)
- [Sparse is Enough in Scaling Transformers](https://mlsanity.com/paper/2111.12763/) (2021)
- [RobustTAD: Robust Time Series Anomaly Detection via Decomposition and Convolutional Neural Networks](https://mlsanity.com/paper/2002.09545/) (2020)
- [Deep Learning with Long Short-Term Memory for Time Series Prediction](https://mlsanity.com/paper/1810.10161/) (2018)
- [A Survey on Efficient Training of Transformers](https://mlsanity.com/paper/2302.01107/) (2023)
- [Long-Term Mobile Traffic Forecasting Using Deep Spatio-Temporal Neural Networks](https://mlsanity.com/paper/1712.08083/) (2017)
- [Cross-modal Knowledge Distillation for Vision-to-Sensor Action Recognition](https://mlsanity.com/paper/2112.01849/) (2021)
- [SSDNet: State Space Decomposition Neural Network for Time Series Forecasting](https://mlsanity.com/paper/2112.10251/) (2021)
- [A Multi-Scale Decomposition MLP-Mixer for Time Series Analysis](https://mlsanity.com/paper/2310.11959/) (2023)
- [TILDE-Q: A Transformation Invariant Loss Function for Time-Series Forecasting](https://mlsanity.com/paper/2210.15050/) (2022)
- [Progressive Cross-modal Knowledge Distillation for Human Action Recognition](https://mlsanity.com/paper/2208.08090/) (2022)
- [Short-term Renewable Energy Forecasting in Greece using Prophet Decomposition and Tree-based Ensembles](https://mlsanity.com/paper/2107.03825/) (2021)
- [Masked Multi-Step Probabilistic Forecasting for Short-to-Mid-Term Electricity Demand](https://mlsanity.com/paper/2302.06818/) (2023)
- [TLNets: Transformation Learning Networks for long-range time-series prediction](https://mlsanity.com/paper/2305.15770/) (2023)
- [Temporal Spatial Decomposition and Fusion Network for Time Series Forecasting](https://mlsanity.com/paper/2210.03122/) (2022)
- [A Novel Method Combines Moving Fronts, Data Decomposition and Deep Learning to Forecast Intricate Time Series](https://mlsanity.com/paper/2303.06394/) (2023)

## Built on by (most cited first, 1 of 1)

- [Super-Linear: A Lightweight Pretrained Mixture of Linear Experts for Time Series Forecasting](https://mlsanity.com/paper/2509.15105/) (2025)

## Abstract (arXiv)

Time series forecasting is a critical task in domains such as energy, finance, and meteorology, where accurate long-term predictions are essential. While Transformer-based models have shown promise in capturing temporal dependencies, their application to extended sequences is limited by computational inefficiencies and limited generalization. In this study, we propose KEDformer, a knowledge extraction-driven framework that integrates seasonal-trend decomposition to address these challenges. KEDformer leverages knowledge extraction methods that focus on the most informative weights within the self-attention mechanism to reduce computational overhead. Additionally, the proposed KEDformer framework decouples time series into seasonal and trend components. This decomposition enhances the model's ability to capture both short-term fluctuations and long-term patterns. Extensive experiments on five public datasets from energy, transportation, and weather domains demonstrate the effectiveness and competitiveness of KEDformer, providing an efficient solution for long-term time series forecasting.