# Decentralized Attention Fails Centralized Signals: Rethinking Transformers for Medical Time Series

arXiv: https://arxiv.org/abs/2602.18473 · HTML: https://mlsanity.com/paper/2602.18473/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 72 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting](https://mlsanity.com/paper/2205.14415/) (2022)
- [Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency](https://mlsanity.com/paper/2206.08496/) (2022)
- [TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables](https://mlsanity.com/paper/2402.19072/) (2024)
- [Deep Time Series Models: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2407.13278/) (2024)
- [Deep learning-based electroencephalography analysis: a systematic review](https://mlsanity.com/paper/1901.05498/) (2019)
- [DUET: Dual Clustering Enhanced Multivariate Time Series Forecasting](https://mlsanity.com/paper/2412.10859/) (2024)
- [Self-Supervised Graph Neural Networks for Improved Electroencephalographic Seizure Analysis](https://mlsanity.com/paper/2104.08336/) (2021)
- [SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusion](https://mlsanity.com/paper/2404.14197/) (2024)
- [Contrast Everything: A Hierarchical Contrastive Framework for Medical Time-Series](https://mlsanity.com/paper/2310.14017/) (2023)
- [A Comprehensive Survey of Deep Learning for Multivariate Time Series Forecasting: A Channel Strategy Perspective](https://mlsanity.com/paper/2502.10721/) (2025)
- [Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classification](https://mlsanity.com/paper/2405.19363/) (2024)
- [A Comprehensive Survey of Deep Learning for Time Series Forecasting: Architectural Diversity and Open Challenges](https://mlsanity.com/paper/2411.05793/) (2024)
- [Revitalizing Multivariate Time Series Forecasting: Learnable Decomposition with Inter-Series Dependencies and Intra-Series Variations Modeling](https://mlsanity.com/paper/2402.12694/) (2024)
- [Multi-resolution Time-Series Transformer for Long-term Forecasting](https://mlsanity.com/paper/2311.04147/) (2023)
- [Evaluating and Steering Modality Preferences in Multimodal Large Language Model](https://mlsanity.com/paper/2505.20977/) (2025)
- [CATS: Enhancing Multivariate Time Series Forecasting by Constructing Auxiliary Time Series as Exogenous Variables](https://mlsanity.com/paper/2403.01673/) (2024)
- [Learning 2D Invariant Affordance Knowledge for 3D Affordance Grounding](https://mlsanity.com/paper/2408.13024/) (2024)
- [Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language Navigation](https://mlsanity.com/paper/2505.20897/) (2025)
- [Hierarchical Deep Learning with Generative Adversarial Network for Automatic Cardiac Diagnosis from ECG Signals](https://mlsanity.com/paper/2210.11408/) (2022)
- [Question-guided Knowledge Graph Re-scoring and Injection for Knowledge Graph Question Answering](https://mlsanity.com/paper/2410.01401/) (2024)
- [MedGNN: Towards Multi-resolution Spatiotemporal Graph Learning for Medical Time Series Classification](https://mlsanity.com/paper/2502.04515/) (2025)
- [Language Model Based Text-to-Audio Generation: Anti-Causally Aligned Collaborative Residual Transformers](https://mlsanity.com/paper/2510.04577/) (2025)

## Built on by (most cited first, 5 of 5)

- [AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discovery](https://mlsanity.com/paper/2604.05550/) (2026)
- [Spectral Priors vs. Attention: Investigating the Utility of Attention Mechanisms in EEG-Based Diagnosis](https://mlsanity.com/paper/2605.15433/) (2026)
- [ViTexSZ: Heterogeneous Vision-Text Knowledge Distillation for EEG Seizure Detection](https://mlsanity.com/paper/2608.21445/) (2026)
- [On the role of the tokenizer in ECG transformer models](https://mlsanity.com/paper/2609.15433/) (2026)
- [Beyond Flattened Tokens: Structure-Preserving EEG Decoding with Reusable TriDim Blocks](https://mlsanity.com/paper/2609.19842/) (2026)

## Abstract (arXiv)

Accurate analysis of medical time series (MedTS) data, such as electroencephalography (EEG) and electrocardiography (ECG), plays a pivotal role in healthcare applications, including the diagnosis of brain and heart diseases. MedTS data typically exhibit two critical patterns: temporal dependencies within individual channels and channel dependencies across multiple channels. While recent advances in deep learning have leveraged Transformer-based models to effectively capture temporal dependencies, they often struggle with modeling channel dependencies. This limitation stems from a structural mismatch: MedTS signals are inherently centralized, whereas the Transformer's attention mechanism is decentralized, making it less effective at capturing global synchronization and unified waveform patterns. To address this mismatch, we propose CoTAR (Core Token Aggregation-Redistribution), a centralized MLP-based module designed to replace decentralized attention. Instead of allowing all tokens to interact directly, as in standard attention, CoTAR introduces a global core token that serves as a proxy to facilitate inter-token interactions, thereby enforcing a centralized aggregation and redistribution strategy. This design not only better aligns with the centralized nature of MedTS signals but also reduces computational complexity from quadratic to linear. Experiments on five benchmarks validate the superiority of our method in both effectiveness and efficiency, achieving up to a 11.6% improvement on the APAVA dataset, while using only 33% of the memory and 20% of the inference time compared to the previous state of the art. Code and all training scripts are available at https://github.com/Levi-Ackman/TeCh.