# A Hybrid Transformer Model for Fake News Detection: Leveraging Bayesian Optimization and Bidirectional Recurrent Unit

arXiv: https://arxiv.org/abs/2502.09097 · HTML: https://mlsanity.com/paper/2502.09097/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 20 references that are in the map.
- First posted on arXiv 13 Feb 2025.

## Builds on (most cited first, 10 of 10)

- [RAG-Instruct: Boosting LLMs with Diverse Retrieval-Augmented Instructions](https://mlsanity.com/paper/2501.00353/) (2024)
- [GreenPLM: Cross-Lingual Transfer of Monolingual Pre-Trained Language Models at Almost No Cost](https://mlsanity.com/paper/2211.06993/) (2022)
- [Uncertainty is Fragile: Manipulating Uncertainty in Large Language Models](https://mlsanity.com/paper/2407.11282/) (2024)
- [Enhancing Document-level Event Argument Extraction with Contextual Clues and Role Relevance](https://mlsanity.com/paper/2310.05991/) (2023)
- [Challenges and Innovations in LLM-Powered Fake News Detection: A Synthesis of Approaches and Future Directions](https://mlsanity.com/paper/2502.00339/) (2025)
- [Improving Academic Skills Assessment with NLP and Ensemble Learning](https://mlsanity.com/paper/2409.19013/) (2024)
- [Optimization of Transformer heart disease prediction model based on particle swarm optimization algorithm](https://mlsanity.com/paper/2412.02801/) (2024)
- [Simultaneously detecting spatiotemporal changes with penalized Poisson regression models](https://mlsanity.com/paper/2405.06613/) (2024)
- [A Hybrid Attention Framework for Fake News Detection with Large Language Models](https://mlsanity.com/paper/2501.11967/) (2025)
- [Detecting COVID-19 Conspiracy Theories with Transformers and TF-IDF](https://mlsanity.com/paper/2205.00377/) (2022)

## Built on by (most cited first, 3 of 3)

- [Unmasking Digital Falsehoods: A Comparative Analysis of LLM-Based Misinformation Detection Strategies](https://mlsanity.com/paper/2503.00724/) (2025)
- [Uncovering the Fragility of Trustworthy LLMs through Chinese Textual Ambiguity](https://mlsanity.com/paper/2507.23121/) (2025)
- [Advancing Sentiment Analysis: A Novel LSTM Framework with Multi-head Attention](https://mlsanity.com/paper/2503.08079/) (2025)

## Abstract (arXiv)

In this paper, we propose an optimized Transformer model that integrates Bayesian algorithms with a Bidirectional Gated Recurrent Unit (BiGRU), and apply it to fake news classification for the first time. First, we employ the TF-IDF method to extract features from news texts and transform them into numeric representations to facilitate subsequent machine learning tasks. Two sets of experiments are then conducted for fake news detection and classification: one using a Transformer model optimized only with BiGRU, and the other incorporating Bayesian algorithms into the BiGRU-based Transformer. Experimental results show that the BiGRU-optimized Transformer achieves 100% accuracy on the training set and 99.67% on the test set, while the addition of the Bayesian algorithm maintains 100% accuracy on the training set and slightly improves test-set accuracy to 99.73%. This indicates that the Bayesian algorithm boosts model accuracy by 0.06%, further enhancing the detection capability for fake news. Moreover, the proposed algorithm converges rapidly at around the 10th training epoch with accuracy nearing 100%, demonstrating both its effectiveness and its fast classification ability. Overall, the optimized Transformer model, enhanced by the Bayesian algorithm and BiGRU, exhibits excellent continuous learning and detection performance, offering a robust technical means to combat the spread of fake news in the current era of information overload.