mlsanityData to 7 Oct 2026

Synthetic Disinformation Attacks on Automated Fact Verification Systems

arXiv 2202.09381 ↗First posted 18 Feb 2022cs.CLcited by 44

Not on the map. Why?

Abstract (arXiv)

Automated fact-checking is a needed technology to curtail the spread of online misinformation. One current framework for such solutions proposes to verify claims by retrieving supporting or refuting evidence from related textual sources. However, the realistic use cases for fact-checkers will require verifying claims against evidence sources that could be affected by the same misinformation. Furthermore, the development of modern NLP tools that can produce coherent, fabricated content would allow malicious actors to systematically generate adversarial disinformation for fact-checkers. In this work, we explore the sensitivity of automated fact-checkers to synthetic adversarial evidence in two simulated settings: AdversarialAddition, where we fabricate documents and add them to the evidence repository available to the fact-checking system, and AdversarialModification, where existing evidence source documents in the repository are automatically altered. Our study across multiple models on three benchmarks demonstrates that these systems suffer significant performance drops against these attacks. Finally, we discuss the growing threat of modern NLG systems as generators of disinformation in the context of the challenges they pose to automated fact-checkers.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 39 of its 56 references that are in the map. First posted on arXiv 18 Feb 2022.

Builds on

39 of 56 references.

References · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  7. 2019Language Models as Knowledge Bases?cited by 1,922
  8. 2016Enriching Word Vectors with Subword Informationcited by 1,578
  9. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  10. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  11. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  12. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
27 more
  1. 2019Defending Against Neural Fake Newscited by 553
  2. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  3. 2021All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Textcited by 287
  4. 2019Automatic Detection of Generated Text is Easiest when Humans are Fooledcited by 260
  5. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 240
  6. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
  7. 2016A Decomposable Attention Model for Natural Language Inferencecited by 214
  8. 2020CORD-19: The COVID-19 Open Research Datasetcited by 179
  9. 2018The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigationcited by 159
  10. 2018Combining Fact Extraction and Verification with Neural Semantic Matching Networkscited by 120
  11. 2019Fine-grained Fact Verification with Kernel Graph Attention Networkcited by 105
  12. 2019GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verificationcited by 100
  13. 2021COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemiccited by 100
  14. 2018Automated Fact Checking: Task formulations, methods and future directionscited by 96
  15. 2019Attention Guided Graph Convolutional Networks for Relation Extractioncited by 82
  16. 2020Coreferential Reasoning Learning for Language Representationcited by 80
  17. 2019Reasoning Over Semantic-Level Graph for Fact Checkingcited by 69
  18. 2021Towards Few-Shot Fact-Checking via Perplexitycited by 63
  19. 2018UKP-Athene: Multi-Sentence Textual Entailment for Claim Verificationcited by 60
  20. 2020Language Models as Fact Checkers?cited by 51
  21. 2021Fool Me Twice: Entailment from Wikipedia Gamificationcited by 50
  22. 2020Scientific Claim Verification with VERT5ERINIcited by 47
  23. 2020A Paragraph-level Multi-task Learning Model for Scientific Fact-Verificationcited by 29
  24. 2020Generating Label Cohesive and Well-Formed Adversarial Claimscited by 27
  25. 2021"I'm Not Mad": Commonsense Implications of Negation and Contradictioncited by 26
  26. 2021A Multi-Level Attention Model for Evidence-Based Fact Checkingcited by 13
  27. 2020Hierarchical Evidence Set Modeling for Automated Fact Extraction and Verificationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 4; 2024: 11; 2025: 14; 2026: 11.

44 papers.

Sort:

By area:

  1. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  2. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
  3. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  4. 2024Certifiably Robust RAG against Retrieval Corruptioncited by 144
  5. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 142
  6. 2024Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communitiescited by 84
  7. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  8. 2024What Evidence Do Language Models Find Convincing?cited by 74
  9. 2024Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineeringcited by 60
  10. 2025SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Modelcited by 31
  11. 2023Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applicationscited by 28
  12. 2025RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defectscited by 27
32 more
  1. 2025DRAGged into Conflicts: Detecting and Addressing Conflicting Sources in Search-Augmented LLMscited by 21
  2. 2026Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systemscited by 16
  3. 2023On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlookcited by 15
  4. 2022Claim-Dissector: An Interpretable Fact-Checking System with Joint Re-ranking and Veracity Predictioncited by 12
  5. 2022Defending Against Disinformation Attacks in Open-Domain Question Answeringcited by 12
  6. 2023A Security Risk Taxonomy for Prompt-Based Interaction With Large Language Modelscited by 12
  7. 2022Fact-Saboteurs: A Taxonomy of Evidence Manipulation Attacks against Fact-Verification Systemscited by 11
  8. 2025TracLLM: A Generic Framework for Attributing Long Context LLMscited by 11
  9. 2026OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAGcited by 11
  10. 2024Toward Robust RALMs: Revealing the Impact of Imperfect Retrieval on Retrieval-Augmented Language Modelscited by 8
  11. 2025Adversarial Attacks Against Automated Fact-Checking: A Surveycited by 5
  12. 2025Fact2Fiction: Targeted Poisoning Attack to Agentic Fact-checking Systemcited by 4
  13. 2025Attacks by Content: Automated Fact-checking is an AI Security Issuecited by 4
  14. 2024Untangle the KNOT: Interweaving Conflicting Knowledge and Reasoning Skills in Large Language Modelscited by 3
  15. 2025CAMOUFLAGE: Exploiting Misinformation Detection Systems Through LLM-driven Adversarial Claim Transformationcited by 3
  16. 2025Through the Stealth Lens: Attention-Aware Defenses Against Poisoning in RAGcited by 3
  17. 2025Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledgecited by 3
  18. 2026Stateless Yet Not Forgetful: Implicit Memory as a Hidden Channel in LLMscited by 3
  19. 2025DisarmRAG: Stealthy Retriever-Centric Poisoning to Disable Self-Correction in Retrieval-Augmented Generation (Extended Version)cited by 2
  20. 2025Tailored untruths: How personalisation challenges LLM safeguardscited by 2
  21. 2026Misinformation Propagation in Benign Multi-Agent Systemscited by 2
  22. 2025Toward a Safer Web: Multilingual Multi-Agent LLMs for Mitigating Adversarial Misinformation Attackscited by 1
  23. 2024Context Quality Matters in Training Fusion-in-Decoder for Extractive Open-Domain Question Answeringcited by 0
  24. 2024How Entangled is Factuality and Deception in German?cited by 0
  25. 2025Mitigating GenAI-Powered Evidence Pollution for Out-Of-Context Misinformation Detectioncited by 0
  26. 2026LogicPoison: Logical Attacks on Graph Retrieval-Augmented Generationcited by 0
  27. 2026AtomEval: Validity-Aware Atomic Evaluation of Adversarial Claim Rewriting in Fact Verificationcited by 0
  28. 2026Agentic Adversarial Rewriting Exposes Architectural Vulnerabilities in Black-Box NLP Pipelinescited by 0
  29. 2026Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerabilitycited by 0
  30. 2026TriShieldRAG: 3 Rings, One Blind Spot in Layered Defenses for Retrieval-Augmented Generationcited by 0
  31. 2026Large Language Models in Resolving Contextual Knowledge Conflictscited by 0
  32. 2026SoK: Formal Methods for Fact-Checking and Information Integritycited by 0