# Optimization of Transformer heart disease prediction model based on particle swarm optimization algorithm

arXiv: https://arxiv.org/abs/2412.02801 · HTML: https://mlsanity.com/paper/2412.02801/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 12 references that are in the map.
- First posted on arXiv 3 Dec 2024.

## Builds on (most cited first, 3 of 3)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Particle Swarm Optimization: A survey of historical and recent developments with hybridization perspectives](https://mlsanity.com/paper/1804.05319/) (2018)
- [Skin Cancer Detection utilizing Deep Learning: Classification of Skin Lesion Images using a Vision Transformer](https://mlsanity.com/paper/2407.18554/) (2024)

## Built on by (most cited first, 5 of 5)

- [Unmasking Digital Falsehoods: A Comparative Analysis of LLM-Based Misinformation Detection Strategies](https://mlsanity.com/paper/2503.00724/) (2025)
- [A Hybrid Transformer Model for Fake News Detection: Leveraging Bayesian Optimization and Bidirectional Recurrent Unit](https://mlsanity.com/paper/2502.09097/) (2025)
- [Advancing Sentiment Analysis: A Novel LSTM Framework with Multi-head Attention](https://mlsanity.com/paper/2503.08079/) (2025)
- [Machine Learning Algorithm for Noise Reduction and Disease-Causing Gene Feature Extraction in Gene Sequencing Data](https://mlsanity.com/paper/2505.19740/) (2025)
- [Residual GRU+MHSA: A Lightweight Hybrid Recurrent Attention Model for Cardiovascular Disease Detection](https://mlsanity.com/paper/2512.14563/) (2025)

## Abstract (arXiv)

Aiming at the latest particle swarm optimization algorithm, this paper proposes an improved Transformer model to improve the accuracy of heart disease prediction and provide a new algorithm idea. We first use three mainstream machine learning classification algorithms - decision tree, random forest and XGBoost, and then output the confusion matrix of these three models. The results showed that the random forest model had the best performance in predicting the classification of heart disease, with an accuracy of 92.2%. Then, we apply the Transformer model based on particle swarm optimization (PSO) algorithm to the same dataset for classification experiment. The results show that the classification accuracy of the model is as high as 96.5%, 4.3 percentage points higher than that of random forest, which verifies the effectiveness of PSO in optimizing Transformer model. From the above research, we can see that particle swarm optimization significantly improves Transformer performance in heart disease prediction. Improving the ability to predict heart disease is a global priority with benefits for all humankind. Accurate prediction can enhance public health, optimize medical resources, and reduce healthcare costs, leading to healthier populations and more productive societies worldwide. This advancement paves the way for more efficient health management and supports the foundation of a healthier, more resilient global community.