# Fake News Detection System using XLNet model with Topic Distributions: CONSTRAINT@AAAI2021 Shared Task

arXiv: https://arxiv.org/abs/2101.11425 · HTML: https://mlsanity.com/paper/2101.11425/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 12 Jan 2021.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [How to Fine-Tune BERT for Text Classification?](https://mlsanity.com/paper/1905.05583/) (2019)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017)
- [Automatic Detection of Fake News](https://mlsanity.com/paper/1708.07104/) (2017)
- [CSI: A Hybrid Deep Model for Fake News Detection](https://mlsanity.com/paper/1703.06959/) (2017)
- [Fighting an Infodemic: COVID-19 Fake News Dataset](https://mlsanity.com/paper/2011.03327/) (2020)
- [MultiFiT: Efficient Multi-lingual Language Model Fine-tuning](https://mlsanity.com/paper/1909.04761/) (2019)
- [A Deep Learning Approach for Automatic Detection of Fake News](https://mlsanity.com/paper/2005.04938/) (2020)
- [NutCracker at WNUT-2020 Task 2: Robustly Identifying Informative COVID-19 Tweets using Ensembling and Adversarial Training](https://mlsanity.com/paper/2010.04335/) (2020)

## Built on by (most cited first, 6 of 6)

- [A Multi-task Model for Sentiment Aided Stance Detection of Climate Change Tweets](https://mlsanity.com/paper/2211.03533/) (2022)
- [A Heuristic-driven Uncertainty based Ensemble Framework for Fake News Detection in Tweets and News Articles](https://mlsanity.com/paper/2104.01791/) (2021)
- [MisRoBAERTa: Transformers versus Misinformation](https://mlsanity.com/paper/2304.07759/) (2023)
- [DRES: Fake news detection by dynamic representation and ensemble selection](https://mlsanity.com/paper/2509.16893/) (2025)
- [A Comparative Study on COVID-19 Fake News Detection Using Different Transformer Based Models](https://mlsanity.com/paper/2208.01355/) (2022)
- [A Multi-Branch Feature Fusion Approach for Health Misinformation Detection and Propagation](https://mlsanity.com/paper/2609.00403/) (2026)

## Abstract (arXiv)

With the ease of access to information, and its rapid dissemination over the internet (both velocity and volume), it has become challenging to filter out truthful information from fake ones. The research community is now faced with the task of automatic detection of fake news, which carries real-world socio-political impact. One such research contribution came in the form of the Constraint@AAA12021 Shared Task on COVID19 Fake News Detection in English. In this paper, we shed light on a novel method we proposed as a part of this shared task. Our team introduced an approach to combine topical distributions from Latent Dirichlet Allocation (LDA) with contextualized representations from XLNet. We also compared our method with existing baselines to show that XLNet + Topic Distributions outperforms other approaches by attaining an F1-score of 0.967.