mlsanityData to 7 Oct 2026

Attacks by Content: Automated Fact-checking is an AI Security Issue

arXiv 2510.11238 ↗First posted 13 Oct 2025cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

When AI agents retrieve and reason over external documents, adversaries can manipulate the data they receive to subvert their behaviour. Previous research has studied indirect prompt injection, where the attacker injects malicious instructions. We argue that injection of instructions is not necessary to manipulate agents - attackers could instead supply biased, misleading, or false information. We term this an attack by content. Existing defenses, which focus on detecting hidden commands, are ineffective against attacks by content. To defend themselves and their users, agents must critically evaluate retrieved information, corroborating claims with external evidence and evaluating source trustworthiness. We argue that this is analogous to an existing NLP task, automated fact-checking, which we propose to repurpose as a cognitive self-defense tool for agents.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 42 of its 104 references that are in the map. First posted on arXiv 13 Oct 2025.

Builds on

42 of 104 references.

References · 42

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  4. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  5. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  6. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  7. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  8. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  9. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  10. 2019Defending Against Neural Fake Newscited by 553
  11. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  12. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
30 more
  1. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  2. 2021A Survey on Automated Fact-Checkingcited by 376
  3. 2023Language Models can Solve Computer Taskscited by 371
  4. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 282
  5. 2019MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claimscited by 169
  6. 2021FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured informationcited by 166
  7. 2023AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Webcited by 144
  8. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 107
  9. 2021COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemiccited by 100
  10. 2020That is a Known Lie: Detecting Previously Fact-Checked Claimscited by 82
  11. 2018Predicting Factuality of Reporting and Bias of News Media Sourcescited by 71
  12. 2019The Limitations of Stylometry for Detecting Machine-Generated Fake Newscited by 63
  13. 2024Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineeringcited by 60
  14. 2018Towards Automated Factchecking: Developing an Annotation Schema and Benchmark for Consistent Automated Claim Detectioncited by 55
  15. 2023Why So Gullible? Enhancing the Robustness of Retrieval-Augmented Models against Counterfactual Noisecited by 52
  16. 2023Multimodal Automated Fact-Checking: A Surveycited by 51
  17. 2022Synthetic Disinformation Attacks on Automated Fact Verification Systemscited by 44
  18. 2020What Was Written vs. Who Read It: News Media Profiling Using Text Analysis and Social Media Contextcited by 42
  19. 2024The Automated Verification of Textual Claims (AVeriTeC) Shared Taskcited by 29
  20. 2025Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checkingcited by 29
  21. 2023Fin-Fact: A Benchmark Dataset for Multimodal Financial Fact Checking and Explanation Generationcited by 25
  22. 2023How susceptible are LLMs to Logical Fallacies?cited by 23
  23. 2023The Intended Uses of Automated Fact-Checking Artefacts: Why, How and Whocited by 19
  24. 2024Teaching Models to Balance Resisting and Accepting Persuasioncited by 19
  25. 2024Designing for Human-Agent Alignment: Understanding what humans want from their agentscited by 18
  26. 2019Tanbih: Get To Know What You Are Readingcited by 16
  27. 2024Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responsescited by 15
  28. 2024HerO at AVeriTeC: The Herd of Open Large Language Models for Verifying Real-World Claimscited by 12
  29. 2024Generating Media Background Checks for Automated Source Critical Reasoningcited by 10
  30. 2024Can LLMs be Scammed? A Baseline Measurement Studycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Latent Causal Void: Explicit Missing-Context Reconstruction for Misinformation Detectioncited by 0
  2. 2026MEDIAREF: A Public Knowledge Store for Media Background Checkscited by 0
  3. 2026PIPES: Securing Agent Perception with Provenance and Priorscited by 0
  4. 2026Lazy Grounding: Attacking Search Agents with Factual Evidencecited by 0