mlsanityData to 7 Oct 2026

Automatic Fake News Detection: Are Models Learning to Reason?

arXiv 2105.07698 ↗First posted 17 May 2021cs.CL cs.CY cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Most fact checking models for automatic fake news detection are based on reasoning: given a claim with associated evidence, the models aim to estimate the claim veracity based on the supporting or refuting content within the evidence. When these models perform well, it is generally assumed to be due to the models having learned to reason over the evidence with regards to the claim. In this paper, we investigate this assumption of reasoning, by exploring the relationship and importance of both claim and evidence. Surprisingly, we find on political fact checking datasets that most often the highest effectiveness is obtained by utilizing only the evidence, as the impact of including the claim is either negligible or harmful to the effectiveness. This highlights an important problem in what constitutes evidence in existing approaches for automatic fake news detection.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 16 of its 26 references that are in the map. First posted on arXiv 17 May 2021.

Builds on

16 of 26 references.

References · 16

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  3. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  4. 2017"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detectioncited by 361
  5. 2019MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claimscited by 169
  6. 2017Automatic Detection of Fake Newscited by 145
  7. 2016Analyzing the Behavior of Visual Question Answering Modelscited by 119
  8. 2018DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learningcited by 100
  9. 2018Automated Fact Checking: Task formulations, methods and future directionscited by 96
  10. 2020That is a Known Lie: Detecting Previously Fact-Checked Claimscited by 82
  11. 2015Natural Language Inference by Tree-Based Convolution and Heuristic Matchingcited by 42
  12. 2018Integrating Stance Detection and Fact Checking in a Unified Corpuscited by 41
4 more
  1. 2017Lexical Features in Coreference Resolution: To be Used With Cautioncited by 12
  2. 2020Time-Aware Evidence Ranking for Fact-Checkingcited by 12
  3. 2019Neural Check-Worthiness Ranking with Weak Supervision: Finding Sentences for Fact-Checkingcited by 9
  4. 2018DeFactoNLP: Fact Verification using Entity Recognition, TFIDF Vector Comparison and Decomposable Attentioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 2; 2024: 1; 2025: 0; 2026: 1.

7 papers.

  1. 2022Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformationcited by 45
  2. 2023Out-of-distribution Evidence-aware Fake News Detection via Dual Adversarial Debiasingcited by 7
  3. 2022Automatic Fake News Detection: Are current models "fact-checking" or "gut-checking"?cited by 3
  4. 2022Time Will Change Things: An Empirical Study on Dynamic Language Understanding in Social Media Classificationcited by 2
  5. 2023Tackling Fake News in Bengali: Unraveling the Impact of Summarization vs. Augmentation on Pre-trained Language Modelscited by 2
  6. 2024How Entangled is Factuality and Deception in German?cited by 0
  7. 2026How Robust Are Automated Fact-Checking Systems? A Cross-Benchmark Evaluationcited by 0