# Transfer Learning from Transformers to Fake News Challenge Stance Detection (FNC-1) Task

arXiv: https://arxiv.org/abs/1910.14353 · HTML: https://mlsanity.com/paper/1910.14353/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2019.

## Built on by (most cited first, 7 of 7)

- [A Survey on Stance Detection for Mis- and Disinformation Identification](https://mlsanity.com/paper/2103.00242/) (2021)
- [The COVMis-Stance dataset: Stance Detection on Twitter for COVID-19 Misinformation](https://mlsanity.com/paper/2204.02000/) (2022)
- [Detecting COVID-19 Conspiracy Theories with Transformers and TF-IDF](https://mlsanity.com/paper/2205.00377/) (2022)
- [ReINTEL Challenge 2020: Exploiting Transfer Learning Models for Reliable Intelligence Identification on Vietnamese Social Network Sites](https://mlsanity.com/paper/2102.10794/) (2021)
- [BaIT: Barometer for Information Trustworthiness](https://mlsanity.com/paper/2206.07535/) (2022)
- [Fine-grained Czech News Article Dataset: An Interdisciplinary Approach to Trustworthiness Analysis](https://mlsanity.com/paper/2212.08550/) (2022)
- [Dataset of News Articles with Provenance Metadata for Media Relevance Assessment](https://mlsanity.com/paper/2506.09847/) (2025)

## Abstract (arXiv)

In this paper, we report improved results of the Fake News Challenge Stage 1 (FNC-1) stance detection task. This gain in performance is due to the generalization power of large language models based on Transformer architecture, invented, trained and publicly released over the last two years. Specifically (1) we improved the FNC-1 best performing model adding BERT sentence embedding of input sequences as a model feature, (2) we fine-tuned BERT, XLNet, and RoBERTa transformers on FNC-1 extended dataset and obtained state-of-the-art results on FNC-1 task.