mlsanityData to 7 Oct 2026

When Claims Evolve: Evaluating and Enhancing the Robustness of Embedding Models Against Misinformation Edits

arXiv 2503.03417 ↗First posted 5 Mar 2025cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Online misinformation remains a critical challenge, and fact-checkers increasingly rely on claim matching systems that use sentence embedding models to retrieve relevant fact-checks. However, as users interact with claims online, they often introduce edits, and it remains unclear whether current embedding models used in retrieval are robust to such edits. To investigate this, we introduce a perturbation framework that generates valid and natural claim variations, enabling us to assess the robustness of a wide-range of sentence embedding models in a multi-stage retrieval pipeline and evaluate the effectiveness of various mitigation approaches. Our evaluation reveals that standard embedding models exhibit notable performance drops on edited claims, while LLM-distilled embedding models offer improved robustness at a higher computational cost. Although a strong reranker helps to reduce the performance drop, it cannot fully compensate for first-stage retrieval gaps. To address these retrieval gaps, we evaluate train- and inference-time mitigation approaches, demonstrating that they can improve in-domain robustness by up to 17 percentage points and boost out-of-domain generalization by 10 percentage points. Overall, our findings provide practical improvements to claim-matching systems, enabling more reliable fact-checking of evolving misinformation. Code and data are available at https://github.com/JabezNzomo99/claim-matching-robustness.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 51 of its 83 references that are in the map. First posted on arXiv 5 Mar 2025.

Builds on

51 of 83 references.

References · 51

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  6. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  7. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  8. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  9. 2019Passage Re-ranking with BERTcited by 629
  10. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  11. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  12. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 466
38 more
  1. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
  2. 2021A Survey on Automated Fact-Checkingcited by 376
  3. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  4. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  5. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  6. 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 271
  7. 2019Document Expansion by Query Predictioncited by 225
  8. 2019Multi-Stage Document Ranking with BERTcited by 200
  9. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  10. 2023RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!cited by 151
  11. 2021Automated Fact-Checking for Assisting Human Fact-Checkerscited by 144
  12. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  13. 2024Best Practices and Lessons Learned on Synthetic Datacited by 104
  14. 2020Reevaluating Adversarial Examples in Natural Languagecited by 83
  15. 2020That is a Known Lie: Detecting Previously Fact-Checked Claimscited by 82
  16. 2022Multi-VALUE: A Framework for Cross-Dialectal English NLPcited by 51
  17. 2020Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Mediacited by 46
  18. 2022Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformationcited by 45
  19. 2021Claim Matching Beyond English to Scale Global Fact-Checkingcited by 42
  20. 2020DeSePtion: Dual Sequence Prediction and Adversarial Examples for Improved Fact-Checkingcited by 26
  21. 2023Scaling Down, LiTting Up: Efficient Zero-Shot Listwise Reranking with Seq2seq Encoder-Decoder Modelscited by 26
  22. 2020MALCOM: Generating Malicious Comments to Attack Neural Fake News Detection Modelscited by 25
  23. 2023When do Generative Query and Document Expansions Fail? A Comprehensive Study Across Methods, Retrievers, and Datasetscited by 25
  24. 2024From Languages to Geographies: Towards Evaluating Cultural Bias in Hate Speech Datasetscited by 23
  25. 2019Fake News Detection via NLP is Vulnerable to Adversarial Attackscited by 19
  26. 2021Assisting the Human Fact-Checkers: Detecting All Previously Fact-Checked Claims in a Documentcited by 19
  27. 2023Beyond Scale: The Diversity Coefficient as a Data Quality Metric for Variability in Natural Language Datacited by 19
  28. 2024NLP Systems That Can't Tell Use from Mention Censor Counterspeech, but Teaching the Distinction Helpscited by 19
  29. 2024Contextual Document Embeddingscited by 19
  30. 2023How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attackscited by 18
  31. 2023From Chaos to Clarity: Claim Normalization to Empower Fact-Checkingcited by 17
  32. 2021Article Reranking by Memory-Enhanced Key Sentence Matching for Detecting Previously Fact-Checked Claimscited by 16
  33. 2022Matching Tweets With Applicable Fact-Checks Across Languagescited by 15
  34. 2023Some Observations on Fact-Checking Work with Implications for Computational Supportcited by 9
  35. 2024NaijaHate: Evaluating Hate Speech Detection on Nigerian Twitter Using Representative Datacited by 9
  36. 2023Verifying the Robustness of Automatic Credibility Assessmentcited by 8
  37. 2023Revealing the Blind Spot of Sentence Encoder Evaluation by HEROScited by 5
  38. 2022An Adversarial Benchmark for Fake News Detection Modelscited by 4

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.