# Evaluating BERT-based Pre-training Language Models for Detecting Misinformation

arXiv: https://arxiv.org/abs/2203.07731 · HTML: https://mlsanity.com/paper/2203.07731/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 62 references that are in the map.
- First posted on arXiv 15 Mar 2022.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [How to Fine-Tune BERT for Text Classification?](https://mlsanity.com/paper/1905.05583/) (2019)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [GCAN: Graph-aware Co-Attention Networks for Explainable Fake News Detection on Social Media](https://mlsanity.com/paper/2004.11648/) (2020)
- [CSI: A Hybrid Deep Model for Fake News Detection](https://mlsanity.com/paper/1703.06959/) (2017)
- [Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threads](https://mlsanity.com/paper/1511.07487/) (2015)
- [Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment Classification](https://mlsanity.com/paper/1908.11860/) (2019)
- [All-in-one: Multi-task Learning for Rumour Verification](https://mlsanity.com/paper/1806.03713/) (2018)
- [Comparing BERT against traditional machine learning text classification](https://mlsanity.com/paper/2005.13012/) (2020)
- [Jointly embedding the local and global relations of heterogeneous graph for rumor detection](https://mlsanity.com/paper/1909.04465/) (2019)
- [The Surprising Performance of Simple Baselines for Misinformation Detection](https://mlsanity.com/paper/2104.06952/) (2021)
- [Fake News Identification on Twitter with Hybrid CNN and RNN Models](https://mlsanity.com/paper/1806.11316/) (2018)
- [DTCA: Decision Tree-based Co-Attention Networks for Explainable Claim Verification](https://mlsanity.com/paper/2004.13455/) (2020)
- [Catching Zika Fever: Application of Crowdsourcing and Machine Learning for Tracking Health Misinformation on Twitter](https://mlsanity.com/paper/1707.03778/) (2017)
- [Adaptive Interaction Fusion Networks for Fake News Detection](https://mlsanity.com/paper/2004.10009/) (2020)
- [Stance Classification for Rumour Analysis in Twitter: Exploiting Affective Information and Conversation Structure](https://mlsanity.com/paper/1901.01911/) (2019)
- [BERT based classification system for detecting rumours on Twitter](https://mlsanity.com/paper/2109.02975/) (2021)
- [Deep Two-path Semi-supervised Learning for Fake News Detection](https://mlsanity.com/paper/1906.05659/) (2019)

## Built on by (most cited first, 2 of 2)

- [Multimodal Sentiment Analysis Based on BERT and ResNet](https://mlsanity.com/paper/2412.03625/) (2024)
- [Rumour Evaluation with Very Large Language Models](https://mlsanity.com/paper/2404.16859/) (2024)

## Abstract (arXiv)

It is challenging to control the quality of online information due to the lack of supervision over all the information posted online. Manual checking is almost impossible given the vast number of posts made on online media and how quickly they spread. Therefore, there is a need for automated rumour detection techniques to limit the adverse effects of spreading misinformation. Previous studies mainly focused on finding and extracting the significant features of text data. However, extracting features is time-consuming and not a highly effective process. This study proposes the BERT- based pre-trained language models to encode text data into vectors and utilise neural network models to classify these vectors to detect misinformation. Furthermore, different language models (LM) ' performance with different trainable parameters was compared. The proposed technique is tested on different short and long text datasets. The result of the proposed technique has been compared with the state-of-the-art techniques on the same datasets. The results show that the proposed technique performs better than the state-of-the-art techniques. We also tested the proposed technique by combining the datasets. The results demonstrated that the large data training and testing size considerably improves the technique's performance.