# FANTF: Fuzzy attention network based transformers for time series analysis

arXiv: https://arxiv.org/abs/2504.00070 · HTML: https://mlsanity.com/paper/2504.00070/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 31 Mar 2025.

## Builds on (most cited first, 16 of 16)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy](https://mlsanity.com/paper/2110.02642/) (2021)
- [The UEA multivariate time series classification archive, 2018](https://mlsanity.com/paper/1811.00075/) (2018)
- [Unsupervised Scalable Representation Learning for Multivariate Time Series](https://mlsanity.com/paper/1901.10738/) (2019)
- [AutoFormer: Searching Transformers for Visual Recognition](https://mlsanity.com/paper/2107.00651/) (2021)
- [Transformers in Time-series Analysis: A Tutorial](https://mlsanity.com/paper/2205.01138/) (2022)
- [Differential Convolutional Fuzzy Time Series Forecasting](https://mlsanity.com/paper/2305.08890/) (2023)

## Built on by (most cited first, 1 of 1)

- [FISformer: Replacing Self-Attention with a Fuzzy Inference System in Transformer Models for Time Series Forecasting](https://mlsanity.com/paper/2603.21724/) (2026)

## Abstract (arXiv)

This paper introduces Fuzzy Attention Network-Based Transformer (FANTF), an approach for time series forecasting, classification, and anomaly detection. FANTF combines fuzzy logic concepts with transformer architectures through a novel fuzzy attention mechanism. The proposed mechanism is inspired by Type-2 fuzzy set theory. It uses a learnable Gaussian uncertainty perturbation to model uncertainty in attention relevance scores. This makes it possible for the model to accurately capture uncertainty in temporal relationships better than normal deterministic attention. Transformers capacity to record complex temporal patterns and multivariate connections is enhanced by FANTF. The approach is both architecture-agnostic and task-agnostic. The proposed fuzzy attention module can be used as a direct replacement for the standard attention layer. To evaluate its effectiveness, FANTF is integrated into five transformer backbones: Transformer, Informer, PatchTST, Crossformer, and iTransformer. The fuzzy attention mechanism produces softer and more diverse attention distributions. It provides uncertainty-aware interpretations through attention heatmap visualizations. Experimental evaluations on some real-life datasets reveal that FANTF enhances the forecasting performance, classification, and anomaly detection tasks over traditional transformer-based models. FANTF achieves statistically significant and competitive performance improvements over standard transformer-based baselines on most benchmark datasets (p < 0.05, Wilcoxon signed-rank test), observed under noisy and non-stationary time series conditions.