# Revitalizing Multivariate Time Series Forecasting: Learnable Decomposition with Inter-Series Dependencies and Intra-Series Variations Modeling

arXiv: https://arxiv.org/abs/2402.12694 · HTML: https://mlsanity.com/paper/2402.12694/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 20 Feb 2024.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting](https://mlsanity.com/paper/2205.14415/) (2022)
- [SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction](https://mlsanity.com/paper/2106.09305/) (2021)
- [Long-term Forecasting with TiDE: Time-series Dense Encoder](https://mlsanity.com/paper/2304.08424/) (2023)
- [FITS: Modeling Time Series with 10k Parameters](https://mlsanity.com/paper/2307.03756/) (2023)
- [Frequency-domain MLPs are More Effective Learners in Time Series Forecasting](https://mlsanity.com/paper/2311.06184/) (2023)
- [Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structures](https://mlsanity.com/paper/2207.01186/) (2022)
- [Spatial-Temporal Identity: A Simple yet Effective Baseline for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2208.05233/) (2022)
- [FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting](https://mlsanity.com/paper/2205.08897/) (2022)
- [FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective](https://mlsanity.com/paper/2311.06190/) (2023)
- [SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling](https://mlsanity.com/paper/2302.00861/) (2023)
- [Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors](https://mlsanity.com/paper/2305.18803/) (2023)
- [The Capacity and Robustness Trade-off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2304.05206/) (2023)
- [BasisFormer: Attention-based Time Series Forecasting with Learnable and Interpretable Basis](https://mlsanity.com/paper/2310.20496/) (2023)

## Built on by (most cited first, 30 of 44)

- [CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patterns](https://mlsanity.com/paper/2409.18479/) (2024)
- [TimeBridge: Non-Stationarity Matters for Long-term Time Series Forecasting](https://mlsanity.com/paper/2410.04442/) (2024)
- [Patch-wise Structural Loss for Time Series Forecasting](https://mlsanity.com/paper/2503.00877/) (2025)
- [Temporal Query Network for Efficient Multivariate Time Series Forecasting](https://mlsanity.com/paper/2505.12917/) (2025)
- [OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain](https://mlsanity.com/paper/2505.08550/) (2025)
- [Bridging Past and Future: Distribution-Aware Alignment for Time Series Forecasting](https://mlsanity.com/paper/2509.14181/) (2025)
- [FreEformer: Frequency Enhanced Transformer for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2501.13989/) (2025)
- [HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting](https://mlsanity.com/paper/2505.17431/) (2025)
- [Improving Time Series Forecasting via Instance-aware Post-hoc Revision](https://mlsanity.com/paper/2505.23583/) (2025)
- [UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2406.04975/) (2024)
- [Question-guided Knowledge Graph Re-scoring and Injection for Knowledge Graph Question Answering](https://mlsanity.com/paper/2410.01401/) (2024)
- [Multi-Modal View Enhanced Large Vision Models for Long-Term Time Series Forecasting](https://mlsanity.com/paper/2505.24003/) (2025)
- [ReFocus: Reinforcing Mid-Frequency and Key-Frequency Modeling for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2502.16890/) (2025)
- [CITRAS: Covariate-Informed Transformer for Time Series Forecasting](https://mlsanity.com/paper/2503.24007/) (2025)
- [Enhancing Multivariate Time Series Forecasting with Global Temporal Retrieval](https://mlsanity.com/paper/2602.10847/) (2026)
- [Decentralized Attention Fails Centralized Signals: Rethinking Transformers for Medical Time Series](https://mlsanity.com/paper/2602.18473/) (2026)
- [LiNo: Advancing Recursive Residual Decomposition of Linear and Nonlinear Patterns for Robust Time Series Forecasting](https://mlsanity.com/paper/2410.17159/) (2024)
- [Why Attention Fails: The Degeneration of Transformers into MLPs in Time Series Forecasting](https://mlsanity.com/paper/2509.20942/) (2025)
- [Ada-FCN: Adaptive Frequency-Coupled Network for fMRI-Based Brain Disorder Classification](https://mlsanity.com/paper/2511.04718/) (2025)
- [Moving Beyond Functional Connectivity: Time-Series Modeling for fMRI-Based Brain Disorder Classification](https://mlsanity.com/paper/2602.08262/) (2026)
- [Time-TK: A Multi-Offset Temporal Interaction Framework Combining Transformer and Kolmogorov-Arnold Networks for Time Series Forecasting](https://mlsanity.com/paper/2602.11190/) (2026)
- [Routing Channel-Patch Dependencies in Time Series Forecasting with Graph Spectral Decomposition](https://mlsanity.com/paper/2603.13702/) (2026)
- [MFRS: A Multi-Frequency Reference Series Approach to Scalable and Accurate Time-Series Forecasting](https://mlsanity.com/paper/2503.08328/) (2025)
- [HGTS-Former: Hierarchical HyperGraph Transformer for Multivariate Time Series Analysis](https://mlsanity.com/paper/2508.02411/) (2025)
- [Inner-Instance Normalization for Time Series Forecasting](https://mlsanity.com/paper/2510.08657/) (2025)
- [EMAformer: Enhancing Transformer through Embedding Armor for Time Series Forecasting](https://mlsanity.com/paper/2511.08396/) (2025)
- [SEED: Spectral Entropy-Guided Evaluation of SpatialTemporal Dependencies for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2512.14718/) (2025)
- [vLinear: A Powerful Linear Model for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2601.13768/) (2026)
- [PAMNet: Cycle-aware Phase-Amplitude Modulation Network for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2605.02938/) (2026)
- [What If We Let Forecasting Forget? A Sparse Bottleneck for Cross-Variable Dependencies](https://mlsanity.com/paper/2605.08289/) (2026)

## Abstract (arXiv)

Predicting multivariate time series is crucial, demanding precise modeling of intricate patterns, including inter-series dependencies and intra-series variations. Distinctive trend characteristics in each time series pose challenges, and existing methods, relying on basic moving average kernels, may struggle with the non-linear structure and complex trends in real-world data. Given that, we introduce a learnable decomposition strategy to capture dynamic trend information more reasonably. Additionally, we propose a dual attention module tailored to capture inter-series dependencies and intra-series variations simultaneously for better time series forecasting, which is implemented by channel-wise self-attention and autoregressive self-attention. To evaluate the effectiveness of our method, we conducted experiments across eight open-source datasets and compared it with the state-of-the-art methods. Through the comparison results, our Leddam (LEarnable Decomposition and Dual Attention Module) not only demonstrates significant advancements in predictive performance, but also the proposed decomposition strategy can be plugged into other methods with a large performance-boosting, from 11.87% to 48.56% MSE error degradation.