mlsanityData to 7 Oct 2026

Defending Against Disinformation Attacks in Open-Domain Question Answering

arXiv 2212.10002 ↗First posted 20 Dec 2022cs.CL cs.IRcited by 12

Not on the map. Why?

Abstract (arXiv)

Recent work in open-domain question answering (ODQA) has shown that adversarial poisoning of the search collection can cause large drops in accuracy for production systems. However, little to no work has proposed methods to defend against these attacks. To do so, we rely on the intuition that redundant information often exists in large corpora. To find it, we introduce a method that uses query augmentation to search for a diverse set of passages that could answer the original question but are less likely to have been poisoned. We integrate these new passages into the model through the design of a novel confidence method, comparing the predicted answer to its appearance in the retrieved contexts (what we call Confidence from Answer Redundancy, i.e. CAR). Together these methods allow for a simple but effective way to defend against poisoning attacks that provides gains of nearly 20% exact match across varying levels of data poisoning/knowledge conflicts.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 20 of its 37 references that are in the map. First posted on arXiv 20 Dec 2022.

Builds on

20 of 37 references.

References · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  5. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  6. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  7. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  8. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  9. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  10. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  11. 2020Selective Question Answering under Domain Shiftcited by 228
  12. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 221
8 more
  1. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 142
  2. 2017Simple and Effective Multi-Paragraph Reading Comprehensioncited by 120
  3. 2020Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attackscited by 109
  4. 2020Imitation Attacks and Defenses for Black-box Machine Translation Systemscited by 78
  5. 2021Improving Question Answering Model Robustness with Synthetic Adversarial Data Generationcited by 77
  6. 2022Synthetic Disinformation Attacks on Automated Fact Verification Systemscited by 44
  7. 2021Pseudo-Relevance Feedback for Multiple Representation Dense Retrievalcited by 31
  8. 2019Domain-agnostic Question-Answering with Adversarial Trainingcited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 4; 2026: 3.

12 papers.

  1. 2024Certifiably Robust RAG against Retrieval Corruptioncited by 144
  2. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  3. 2025Retrieval-Augmented Generation with Conflicting Evidencecited by 60
  4. 2024WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipediacited by 43
  5. 2024CrAM: Credibility-Aware Attention Modification in LLMs for Combating Misinformation in RAGcited by 22
  6. 2024Retrieval-Augmented Generation with Estimation of Source Reliabilitycited by 19
  7. 2026OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAGcited by 11
  8. 2025Exploring the Impact of Instruction-Tuning on LLM's Susceptibility to Misinformationcited by 4
  9. 2025Through the Stealth Lens: Attention-Aware Defenses Against Poisoning in RAGcited by 3
  10. 2025HealthContradict: Evaluating Biomedical Knowledge Conflicts in Language Modelscited by 3
  11. 2026LLM-Oriented Information Retrieval: A Denoising-First Perspectivecited by 1
  12. 2026Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inferencecited by 1