# Examining the Limitations of Computational Rumor Detection Models Trained on Static Datasets

arXiv: https://arxiv.org/abs/2309.11576 · HTML: https://mlsanity.com/paper/2309.11576/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 49 references that are in the map.
- First posted on arXiv 20 Sep 2023.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017)
- [Rumor Detection on Social Media with Bi-Directional Graph Convolutional Networks](https://mlsanity.com/paper/2001.06362/) (2020)
- [GCAN: Graph-aware Co-Attention Networks for Explainable Fake News Detection on Social Media](https://mlsanity.com/paper/2004.11648/) (2020)
- [Detection and Resolution of Rumours in Social Media: A Survey](https://mlsanity.com/paper/1704.00656/) (2017)
- [The science of fake news](https://mlsanity.com/paper/2307.07903/) (2023)
- [CoAID: COVID-19 Healthcare Misinformation Dataset](https://mlsanity.com/paper/2006.00885/) (2020)
- [We Need to Talk About Random Splits](https://mlsanity.com/paper/2005.00636/) (2020)
- [Towards generalisable hate speech detection: a review on obstacles and solutions](https://mlsanity.com/paper/2102.08886/) (2021)
- [Detect Rumors in Microblog Posts for Low-Resource Domains via Adversarial Contrastive Learning](https://mlsanity.com/paper/2204.08143/) (2022)
- [Rumor Detection on Twitter with Claim-Guided Hierarchical Graph Attention Networks](https://mlsanity.com/paper/2110.04522/) (2021)
- [Improved Multi-label Classification under Temporal Concept Drift: Rethinking Group-Robust Algorithms in a Label-Wise Setting](https://mlsanity.com/paper/2203.07856/) (2022)
- [Learn over Past, Evolve for Future: Forecasting Temporal Trends for Fake News Detection](https://mlsanity.com/paper/2306.14728/) (2023)
- [It's about Time: Rethinking Evaluation on Rumor Detection Benchmarks using Chronological Splits](https://mlsanity.com/paper/2302.03147/) (2023)
- [Bringing replication and reproduction together with generalisability in NLP: Three reproduction studies for Target Dependent Sentiment Analysis](https://mlsanity.com/paper/1806.05219/) (2018)
- [Examining Temporalities on Stance Detection towards COVID-19 Vaccination](https://mlsanity.com/paper/2304.04806/) (2023)

## Built on by (most cited first, 2 of 2)

- [Toward Verifiable Misinformation Detection: A Multi-Tool LLM Agent Framework](https://mlsanity.com/paper/2508.03092/) (2025)
- [RAGAT-Mind: A Multi-Granular Modeling Approach for Rumor Detection Based on MindSpore](https://mlsanity.com/paper/2504.17574/) (2025)

## Abstract (arXiv)

A crucial aspect of a rumor detection model is its ability to generalize, particularly its ability to detect emerging, previously unknown rumors. Past research has indicated that content-based (i.e., using solely source posts as input) rumor detection models tend to perform less effectively on unseen rumors. At the same time, the potential of context-based models remains largely untapped. The main contribution of this paper is in the in-depth evaluation of the performance gap between content and context-based models specifically on detecting new, unseen rumors. Our empirical findings demonstrate that context-based models are still overly dependent on the information derived from the rumors' source post and tend to overlook the significant role that contextual information can play. We also study the effect of data split strategies on classifier performance. Based on our experimental results, the paper also offers practical suggestions on how to minimize the effects of temporal concept drift in static datasets during the training of rumor detection methods.