# Taking a Stance on Fake News: Towards Automatic Disinformation Assessment via Deep Bidirectional Transformer Language Models for Stance Detection

arXiv: https://arxiv.org/abs/1911.11951 · HTML: https://mlsanity.com/paper/1911.11951/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Nov 2019.

## Built on by (most cited first, 6 of 6)

- [Lost in Translation: Large Language Models in Non-English Content Analysis](https://mlsanity.com/paper/2306.07377/) (2023)
- [Limited Effectiveness of LLM-based Data Augmentation for COVID-19 Misinformation Stance Detection](https://mlsanity.com/paper/2503.02328/) (2025)
- [TATA: Stance Detection via Topic-Agnostic and Topic-Aware Embeddings](https://mlsanity.com/paper/2310.14450/) (2023)
- [Beyond Generative Artificial Intelligence: Roadmap for Natural Language Generation](https://mlsanity.com/paper/2407.10554/) (2024)
- [Human-in-the-Loop Disinformation Detection: Stance, Sentiment, or Something Else?](https://mlsanity.com/paper/2111.05139/) (2021)
- [Angry but Accurate: Detecting and Profiling the Counter-Misinformation Ecosystem on Twitter](https://mlsanity.com/paper/2607.02900/) (2026)

## Abstract (arXiv)

The exponential rise of social media and digital news in the past decade has had the unfortunate consequence of escalating what the United Nations has called a global topic of concern: the growing prevalence of disinformation. Given the complexity and time-consuming nature of combating disinformation through human assessment, one is motivated to explore harnessing AI solutions to automatically assess news articles for the presence of disinformation. A valuable first step towards automatic identification of disinformation is stance detection, where given a claim and a news article, the aim is to predict if the article agrees, disagrees, takes no position, or is unrelated to the claim. Existing approaches in literature have largely relied on hand-engineered features or shallow learned representations (e.g., word embeddings) to encode the claim-article pairs, which can limit the level of representational expressiveness needed to tackle the high complexity of disinformation identification. In this work, we explore the notion of harnessing large-scale deep bidirectional transformer language models for encoding claim-article pairs in an effort to construct state-of-the-art stance detection geared for identifying disinformation. Taking advantage of bidirectional cross-attention between claim-article pairs via pair encoding with self-attention, we construct a large-scale language model for stance detection by performing transfer learning on a RoBERTa deep bidirectional transformer language model, and were able to achieve state-of-the-art performance (weighted accuracy of 90.01%) on the Fake News Challenge Stage 1 (FNC-I) benchmark. These promising results serve as motivation for harnessing such large-scale language models as powerful building blocks for creating effective AI solutions to combat disinformation.