mlsanityData to 7 Oct 2026

An Adversarial Benchmark for Fake News Detection Models

arXiv 2201.00912 ↗First posted 3 Jan 2022cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

With the proliferation of online misinformation, fake news detection has gained importance in the artificial intelligence community. In this paper, we propose an adversarial benchmark that tests the ability of fake news detectors to reason about real-world facts. We formulate adversarial attacks that target three aspects of "understanding": compositional semantics, lexical relations, and sensitivity to modifiers. We test our benchmark using BERT classifiers fine-tuned on the LIAR arXiv:arch-ive/1705648 and Kaggle Fake-News datasets, and show that both models fail to respond to changes in compositional and lexical meaning. Our results strengthen the need for such models to be used in conjunction with other fact checking methods.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 13 of its 28 references that are in the map. First posted on arXiv 3 Jan 2022.

Builds on

13 of 28 references.

References · 13

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  3. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  4. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  5. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 376
  6. 2017"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detectioncited by 361
  7. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 341
  8. 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 295
  9. 2019Combating Adversarial Misspellings with Robust Word Recognitioncited by 162
  10. 2017CSI: A Hybrid Deep Model for Fake News Detectioncited by 101
  11. 2018TI-CNN: Convolutional Neural Networks for Fake News Detectioncited by 22
  12. 2019Fake News Detection via NLP is Vulnerable to Adversarial Attackscited by 19
1 more
  1. 2021How Vulnerable Are Automatic Fake News Detection Methods to Adversarial Attacks?cited by 7

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 1.

4 papers.

  1. 2023Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4cited by 45
  2. 2024Combining Confidence Elicitation and Sample-based Methods for Uncertainty Quantification in Misinformation Mitigationcited by 19
  3. 2023Verifying the Robustness of Automatic Credibility Assessmentcited by 8
  4. 2025When Claims Evolve: Evaluating and Enhancing the Robustness of Embedding Models Against Misinformation Editscited by 3