# Fake News Detection with Heterogeneous Transformer

arXiv: https://arxiv.org/abs/2205.03100 · HTML: https://mlsanity.com/paper/2205.03100/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 6 May 2022.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [LINE: Large-scale Information Network Embedding](https://mlsanity.com/paper/1503.03578/) (2015)
- [Heterogeneous Graph Transformer](https://mlsanity.com/paper/2003.01332/) (2020)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017)
- [FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media](https://mlsanity.com/paper/1809.01286/) (2018)
- [Rumor Detection on Social Media with Bi-Directional Graph Convolutional Networks](https://mlsanity.com/paper/2001.06362/) (2020)
- [GCAN: Graph-aware Co-Attention Networks for Explainable Fake News Detection on Social Media](https://mlsanity.com/paper/2004.11648/) (2020)
- [Exploiting Multi-domain Visual Information for Fake News Detection](https://mlsanity.com/paper/1908.04472/) (2019)
- [Jointly embedding the local and global relations of heterogeneous graph for rumor detection](https://mlsanity.com/paper/1909.04465/) (2019)
- [Adversarial Active Learning based Heterogeneous Graph Neural Network for Fake News Detection](https://mlsanity.com/paper/2101.11206/) (2021)
- [COBRA: Contrastive Bi-Modal Representation Algorithm](https://mlsanity.com/paper/2005.03687/) (2020)
- [Suspicious Massive Registration Detection via Dynamic Heterogeneous Graph Neural Networks](https://mlsanity.com/paper/2012.10831/) (2020)

## Built on by (most cited first, 4 of 4)

- [KERAG: Knowledge-Enhanced Retrieval-Augmented Generation for Advanced Question Answering](https://mlsanity.com/paper/2509.04716/) (2025)
- [Exploring Fake News Detection with Heterogeneous Social Media Context Graphs](https://mlsanity.com/paper/2212.06560/) (2022)
- [Challenges in Pre-Training Graph Neural Networks for Context-Based Fake News Detection: An Evaluation of Current Strategies and Resource Limitations](https://mlsanity.com/paper/2402.18179/) (2024)
- [Performance Analysis of Transformer Based Models (BERT, ALBERT and RoBERTa) in Fake News Detection](https://mlsanity.com/paper/2308.04950/) (2023)

## Abstract (arXiv)

The dissemination of fake news on social networks has drawn public need for effective and efficient fake news detection methods. Generally, fake news on social networks is multi-modal and has various connections with other entities such as users and posts. The heterogeneity in both news content and the relationship with other entities in social networks brings challenges to designing a model that comprehensively captures the local multi-modal semantics of entities in social networks and the global structural representation of the propagation patterns, so as to classify fake news effectively and accurately. In this paper, we propose a novel Transformer-based model: HetTransformer to solve the fake news detection problem on social networks, which utilises the encoder-decoder structure of Transformer to capture the structural information of news propagation patterns. We first capture the local heterogeneous semantics of news, post, and user entities in social networks. Then, we apply Transformer to capture the global structural representation of the propagation patterns in social networks for fake news detection. Experiments on three real-world datasets demonstrate that our model is able to outperform the state-of-the-art baselines in fake news detection.