# When Claims Evolve: Evaluating and Enhancing the Robustness of Embedding Models Against Misinformation Edits

arXiv: https://arxiv.org/abs/2503.03417 · HTML: https://mlsanity.com/paper/2503.03417/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 83 references that are in the map.
- First posted on arXiv 5 Mar 2025.

## Builds on (most cited first, 30 of 51)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale](https://mlsanity.com/paper/2406.17557/) (2024)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://mlsanity.com/paper/2405.17428/) (2024)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [One Embedder, Any Task: Instruction-Finetuned Text Embeddings](https://mlsanity.com/paper/2212.09741/) (2022)
- [BAE: BERT-based Adversarial Examples for Text Classification](https://mlsanity.com/paper/2004.01970/) (2020)
- [Fine-Tuning LLaMA for Multi-Stage Text Retrieval](https://mlsanity.com/paper/2310.08319/) (2023)
- [Document Expansion by Query Prediction](https://mlsanity.com/paper/1904.08375/) (2019)
- [Multi-Stage Document Ranking with BERT](https://mlsanity.com/paper/1910.14424/) (2019)
- [Can LLM-Generated Misinformation Be Detected?](https://mlsanity.com/paper/2309.13788/) (2023)
- [RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!](https://mlsanity.com/paper/2312.02724/) (2023)
- [Automated Fact-Checking for Assisting Human Fact-Checkers](https://mlsanity.com/paper/2103.07769/) (2021)
- [Measure and Improve Robustness in NLP Models: A Survey](https://mlsanity.com/paper/2112.08313/) (2021)
- [Best Practices and Lessons Learned on Synthetic Data](https://mlsanity.com/paper/2404.07503/) (2024)
- [Reevaluating Adversarial Examples in Natural Language](https://mlsanity.com/paper/2004.14174/) (2020)
- [That is a Known Lie: Detecting Previously Fact-Checked Claims](https://mlsanity.com/paper/2005.06058/) (2020)
- [Multi-VALUE: A Framework for Cross-Dialectal English NLP](https://mlsanity.com/paper/2212.08011/) (2022)
- [Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media](https://mlsanity.com/paper/2007.07997/) (2020)
- [Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformation](https://mlsanity.com/paper/2210.13865/) (2022)

## Built on by (most cited first, 3 of 3)

- [Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Models](https://mlsanity.com/paper/2508.03860/) (2025)
- [SAGE: A Realistic Benchmark for Semantic Understanding](https://mlsanity.com/paper/2509.21310/) (2025)
- [The Facade of Truth: Uncovering and Mitigating LLM Susceptibility to Deceptive Evidence](https://mlsanity.com/paper/2601.05478/) (2026)

## Abstract (arXiv)

Online misinformation remains a critical challenge, and fact-checkers increasingly rely on claim matching systems that use sentence embedding models to retrieve relevant fact-checks. However, as users interact with claims online, they often introduce edits, and it remains unclear whether current embedding models used in retrieval are robust to such edits. To investigate this, we introduce a perturbation framework that generates valid and natural claim variations, enabling us to assess the robustness of a wide-range of sentence embedding models in a multi-stage retrieval pipeline and evaluate the effectiveness of various mitigation approaches. Our evaluation reveals that standard embedding models exhibit notable performance drops on edited claims, while LLM-distilled embedding models offer improved robustness at a higher computational cost. Although a strong reranker helps to reduce the performance drop, it cannot fully compensate for first-stage retrieval gaps. To address these retrieval gaps, we evaluate train- and inference-time mitigation approaches, demonstrating that they can improve in-domain robustness by up to 17 percentage points and boost out-of-domain generalization by 10 percentage points. Overall, our findings provide practical improvements to claim-matching systems, enabling more reliable fact-checking of evolving misinformation. Code and data are available at https://github.com/JabezNzomo99/claim-matching-robustness.