# TSLANet: Rethinking Transformers for Time Series Representation Learning

arXiv: https://arxiv.org/abs/2404.08472 · HTML: https://mlsanity.com/paper/2404.08472/

## Key facts

- Cited by 74 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 45 references that are in the map.
- First posted on arXiv 12 Apr 2024.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases](https://mlsanity.com/paper/2103.10697/) (2021)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding](https://mlsanity.com/paper/1802.04431/) (2018)
- [Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy](https://mlsanity.com/paper/2110.02642/) (2021)
- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels](https://mlsanity.com/paper/1910.13051/) (2019)
- [SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction](https://mlsanity.com/paper/2106.09305/) (2021)
- [Global Filter Networks for Image Classification](https://mlsanity.com/paper/2107.00645/) (2021)
- [UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learning](https://mlsanity.com/paper/2201.04676/) (2022)
- [Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structures](https://mlsanity.com/paper/2207.01186/) (2022)
- [Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusion](https://mlsanity.com/paper/2202.04770/) (2022)
- [MHCCL: Masked Hierarchical Cluster-Wise Contrastive Learning for Multivariate Time Series](https://mlsanity.com/paper/2212.01141/) (2022)
- [WFTNet: Exploiting Global and Local Periodicity in Long-term Time Series Forecasting](https://mlsanity.com/paper/2309.11319/) (2023)
- [Temporal Convolutional Explorer Helps Understand 1D-CNN's Learning Behavior in Time Series Classification from Frequency Domain](https://mlsanity.com/paper/2310.05467/) (2023)

## Built on by (most cited first, 30 of 74)

- [Deep Time Series Models: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2407.13278/) (2024)
- [Hierarchical Classification Auxiliary Network for Time Series Forecasting](https://mlsanity.com/paper/2405.18975/) (2024)
- [Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift](https://mlsanity.com/paper/2410.09836/) (2024)
- [Bridging Past and Future: Distribution-Aware Alignment for Time Series Forecasting](https://mlsanity.com/paper/2509.14181/) (2025)
- [TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learning](https://mlsanity.com/paper/2506.13705/) (2025)
- [PPGF: Probability Pattern-Guided Time Series Forecasting](https://mlsanity.com/paper/2502.12802/) (2025)
- [FilterTS: Comprehensive Frequency Filtering for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2505.04158/) (2025)
- [KANICE: Kolmogorov-Arnold Networks with Interactive Convolutional Elements](https://mlsanity.com/paper/2410.17172/) (2024)
- [TSCMamba: Mamba Meets Multi-View Learning for Time Series Classification](https://mlsanity.com/paper/2406.04419/) (2024)
- [Deep Learning for Time Series Forecasting: A Survey](https://mlsanity.com/paper/2503.10198/) (2025)
- [Time Series Analysis in Frequency Domain: A Survey of Open Challenges, Opportunities and Benchmarks](https://mlsanity.com/paper/2504.07099/) (2025)
- [Learning Soft Sparse Shapes for Efficient Time-Series Classification](https://mlsanity.com/paper/2505.06892/) (2025)
- [ClimateLLM: Efficient Weather Forecasting via Frequency-Aware Large Language Models](https://mlsanity.com/paper/2502.11059/) (2025)
- [OTIS: Learning High-Quality Time Series Features With Tiny Encoders](https://mlsanity.com/paper/2410.07299/) (2024)
- [Machine-learning for photoplethysmography analysis: Benchmarking feature, image, and signal-based approaches](https://mlsanity.com/paper/2502.19949/) (2025)
- [LinFormer: A Linear-based Lightweight Transformer Architecture For Time-Aware MIMO Channel Prediction](https://mlsanity.com/paper/2410.21351/) (2024)
- [ReFocus: Reinforcing Mid-Frequency and Key-Frequency Modeling for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2502.16890/) (2025)
- [MoPFormer: Motion-Primitive Transformer for Wearable-Sensor Activity Recognition](https://mlsanity.com/paper/2505.20744/) (2025)
- [Mask the Redundancy: Evolving Masking Representation Learning for Multivariate Time-Series Clustering](https://mlsanity.com/paper/2511.17008/) (2025)
- [From Observations to States: Latent Time Series Forecasting](https://mlsanity.com/paper/2602.00297/) (2026)
- [Peri-midFormer: Periodic Pyramid Transformer for Time Series Analysis](https://mlsanity.com/paper/2411.04554/) (2024)
- [FTMixer: Frequency and Time Domain Representations Fusion for Time Series Modeling](https://mlsanity.com/paper/2405.15256/) (2024)
- [QT-TDM: Planning With Transformer Dynamics Model and Autoregressive Q-Learning](https://mlsanity.com/paper/2407.18841/) (2024)
- [Unveiling the Potential of Text in High-Dimensional Time Series Forecasting](https://mlsanity.com/paper/2501.07048/) (2025)
- [Multi-Order Wavelet Derivative Transform for Deep Time Series Forecasting](https://mlsanity.com/paper/2505.11781/) (2025)
- [MambaSL: Exploring Single-Layer Mamba for Time Series Classification](https://mlsanity.com/paper/2604.15174/) (2026)
- [DeepFRC: An End-to-End Deep Learning Model for Functional Registration and Classification](https://mlsanity.com/paper/2501.18116/) (2025)
- [UniDyG: A Unified and Effective Representation Learning Approach for Large Dynamic Graphs](https://mlsanity.com/paper/2502.16431/) (2025)
- [UniFault: A Fault Diagnosis Foundation Model from Bearing Data](https://mlsanity.com/paper/2504.01373/) (2025)
- [A Unified Shape-Aware Foundation Model for Time Series Classification](https://mlsanity.com/paper/2601.06429/) (2026)

## Abstract (arXiv)

Time series data, characterized by its intrinsic long and short-range dependencies, poses a unique challenge across analytical applications. While Transformer-based models excel at capturing long-range dependencies, they face limitations in noise sensitivity, computational efficiency, and overfitting with smaller datasets. In response, we introduce a novel Time Series Lightweight Adaptive Network (TSLANet), as a universal convolutional model for diverse time series tasks. Specifically, we propose an Adaptive Spectral Block, harnessing Fourier analysis to enhance feature representation and to capture both long-term and short-term interactions while mitigating noise via adaptive thresholding. Additionally, we introduce an Interactive Convolution Block and leverage self-supervised learning to refine the capacity of TSLANet for decoding complex temporal patterns and improve its robustness on different datasets. Our comprehensive experiments demonstrate that TSLANet outperforms state-of-the-art models in various tasks spanning classification, forecasting, and anomaly detection, showcasing its resilience and adaptability across a spectrum of noise levels and data sizes. The code is available at https://github.com/emadeldeen24/TSLANet.