# CARD: Channel Aligned Robust Blend Transformer for Time Series Forecasting

arXiv: https://arxiv.org/abs/2305.12095 · HTML: https://mlsanity.com/paper/2305.12095/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 62 references that are in the map.
- First posted on arXiv 20 May 2023.

## Builds on (most cited first, 30 of 44)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting](https://mlsanity.com/paper/2205.14415/) (2022)
- [XCiT: Cross-Covariance Image Transformers](https://mlsanity.com/paper/2106.09681/) (2021)
- [Long-term Forecasting with TiDE: Time-series Dense Encoder](https://mlsanity.com/paper/2304.08424/) (2023)
- [TSMixer: An All-MLP Architecture for Time Series Forecasting](https://mlsanity.com/paper/2303.06053/) (2023)
- [A Multi-Horizon Quantile Recurrent Forecaster](https://mlsanity.com/paper/1711.11053/) (2017)

## Built on by (most cited first, 30 of 55)

- [Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts](https://mlsanity.com/paper/2409.16040/) (2024)
- [Deep Time Series Models: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2407.13278/) (2024)
- [CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuning](https://mlsanity.com/paper/2403.07300/) (2024)
- [Kolmogorov-Arnold Networks for Time Series: Bridging Predictive Power and Interpretability](https://mlsanity.com/paper/2406.02496/) (2024)
- [A Comprehensive Survey of Deep Learning for Multivariate Time Series Forecasting: A Channel Strategy Perspective](https://mlsanity.com/paper/2502.10721/) (2025)
- [Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classification](https://mlsanity.com/paper/2405.19363/) (2024)
- [From Similarity to Superiority: Channel Clustering for Time Series Forecasting](https://mlsanity.com/paper/2404.01340/) (2024)
- [Rethinking Channel Dependence for Multivariate Time Series Forecasting: Learning from Leading Indicators](https://mlsanity.com/paper/2401.17548/) (2024)
- [Universal Time-Series Representation Learning: A Survey](https://mlsanity.com/paper/2401.03717/) (2024)
- [OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain](https://mlsanity.com/paper/2505.08550/) (2025)
- [Scaling-laws for Large Time-series Models](https://mlsanity.com/paper/2405.13867/) (2024)
- [FreEformer: Frequency Enhanced Transformer for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2501.13989/) (2025)
- [On the Identification of Temporally Causal Representation with Instantaneous Dependence](https://mlsanity.com/paper/2405.15325/) (2024)
- [Time Evidence Fusion Network: Multi-source View in Long-Term Time Series Forecasting](https://mlsanity.com/paper/2405.06419/) (2024)
- [MAESTRO : Adaptive Sparse Attention and Robust Learning for Multimodal Dynamic Time Series](https://mlsanity.com/paper/2509.25278/) (2025)
- [Learning General Causal Structures with Hidden Dynamic Process for Climate Analysis](https://mlsanity.com/paper/2501.12500/) (2025)
- [S4M: S4 for multivariate time series forecasting with Missing values](https://mlsanity.com/paper/2503.00900/) (2025)
- [CITRAS: Covariate-Informed Transformer for Time Series Forecasting](https://mlsanity.com/paper/2503.24007/) (2025)
- [Loss Shaping Constraints for Long-Term Time Series Forecasting](https://mlsanity.com/paper/2402.09373/) (2024)
- [Not All Data are Good Labels: On the Self-supervised Labeling for Time Series Forecasting](https://mlsanity.com/paper/2502.14704/) (2025)
- [Multi-Order Wavelet Derivative Transform for Deep Time Series Forecasting](https://mlsanity.com/paper/2505.11781/) (2025)
- [Baguan-TS: A Sequence-Native In-Context Learning Model for Time Series Forecasting with Covariates](https://mlsanity.com/paper/2603.17439/) (2026)
- [DisenTS: Disentangled Channel Evolving Pattern Modeling for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2410.22981/) (2024)
- [Dataset-Driven Channel Masks in Transformers for Multivariate Time Series](https://mlsanity.com/paper/2410.23222/) (2024)
- [Why Attention Fails: The Degeneration of Transformers into MLPs in Time Series Forecasting](https://mlsanity.com/paper/2509.20942/) (2025)
- [Goal-Oriented Time-Series Forecasting: Foundation Framework Design](https://mlsanity.com/paper/2504.17493/) (2025)
- [S2Transformer: Scalable Structured Transformers for Global Station Weather Forecasting](https://mlsanity.com/paper/2509.19648/) (2025)
- [CPiRi: Channel Permutation-Invariant Relational Interaction for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2601.20318/) (2026)
- [Robust Time Series Forecasting with Non-Heavy-Tailed Gaussian Loss-Weighted Sampler](https://mlsanity.com/paper/2406.13871/) (2024)
- [Channel Matters: Estimating Channel Influence for Multivariate Time Series](https://mlsanity.com/paper/2408.14763/) (2024)

## Abstract (arXiv)

Recent studies have demonstrated the great power of Transformer models for time series forecasting. One of the key elements that lead to the transformer's success is the channel-independent (CI) strategy to improve the training robustness. However, the ignorance of the correlation among different channels in CI would limit the model's forecasting capacity. In this work, we design a special Transformer, i.e., Channel Aligned Robust Blend Transformer (CARD for short), that addresses key shortcomings of CI type Transformer in time series forecasting. First, CARD introduces a channel-aligned attention structure that allows it to capture both temporal correlations among signals and dynamical dependence among multiple variables over time. Second, in order to efficiently utilize the multi-scale knowledge, we design a token blend module to generate tokens with different resolutions. Third, we introduce a robust loss function for time series forecasting to alleviate the potential overfitting issue. This new loss function weights the importance of forecasting over a finite horizon based on prediction uncertainties. Our evaluation of multiple long-term and short-term forecasting datasets demonstrates that CARD significantly outperforms state-of-the-art time series forecasting methods. The code is available at the following repository:https://github.com/wxie9/CARD