# Hoaxpedia: A Unified Wikipedia Hoax Articles Dataset

arXiv: https://arxiv.org/abs/2405.02175 · HTML: https://mlsanity.com/paper/2405.02175/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 68 references that are in the map.
- First posted on arXiv 3 May 2024.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation](https://mlsanity.com/paper/2010.16061/) (2020)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims](https://mlsanity.com/paper/1909.03242/) (2019)
- [The Fact Extraction and VERification (FEVER) Shared Task](https://mlsanity.com/paper/1811.10971/) (2018)
- [Detection and Resolution of Rumours in Social Media: A Survey](https://mlsanity.com/paper/1704.00656/) (2017)
- [SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours](https://mlsanity.com/paper/1704.05972/) (2017)
- [Where Are the Facts? Searching for Fact-checked Information to Alleviate the Spread of Fake News](https://mlsanity.com/paper/2010.03159/) (2020)
- [Controlled Analyses of Social Biases in Wikipedia Bios](https://mlsanity.com/paper/2101.00078/) (2020)
- [DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformation](https://mlsanity.com/paper/2205.12617/) (2022)
- [VEWS: A Wikipedia Vandal Early Warning System](https://mlsanity.com/paper/1507.01272/) (2015)
- [Language-agnostic Topic Classification for Wikipedia](https://mlsanity.com/paper/2103.00068/) (2021)
- [Wiki-Reliability: A Large Scale Dataset for Content Reliability on Wikipedia](https://mlsanity.com/paper/2105.04117/) (2021)
- [Analysing State-Backed Propaganda Websites: a New Dataset and Linguistic Study](https://mlsanity.com/paper/2310.14032/) (2023)
- [Wikipedia Vandal Early Detection: from User Behavior to User Embedding](https://mlsanity.com/paper/1706.00887/) (2017)
- [Fair multilingual vandalism detection system for Wikipedia](https://mlsanity.com/paper/2306.01650/) (2023)
- [Orphan Articles: The Dark Matter of Wikipedia](https://mlsanity.com/paper/2306.03940/) (2023)
- [The role of online attention in the supply of disinformation in Wikipedia](https://mlsanity.com/paper/2302.08576/) (2023)

## Abstract (arXiv)

Hoaxes are a recognised form of disinformation created deliberately, with potential serious implications in the credibility of reference knowledge resources such as Wikipedia. What makes detecting Wikipedia hoaxes hard is that they often are written according to the official style guidelines. In this work, we first provide a systematic analysis of similarities and discrepancies between legitimate and hoax Wikipedia articles, and introduce Hoaxpedia, a collection of 311 hoax articles (from existing literature and official Wikipedia lists), together with semantically similar legitimate articles, which together form a binary text classification dataset aimed at fostering research in automated hoax detection. In this paper, We report results after analyzing several language models, hoax-to-legit ratios, and the amount of text classifiers are exposed to (full article vs the article's definition alone). Our results suggest that detecting deceitful content in Wikipedia based on content alone is hard but feasible, and complement our analysis with a study on the differences in distributions in edit histories, and find that looking at this feature yields better classification results than context.