# CrossAug: A Contrastive Data Augmentation Method for Debiasing Fact Verification Models

arXiv: https://arxiv.org/abs/2109.15107 · HTML: https://mlsanity.com/paper/2109.15107/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 19 references that are in the map.
- First posted on arXiv 30 Sep 2021.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Annotation Artifacts in Natural Language Inference Data](https://mlsanity.com/paper/1803.02324/) (2018)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://mlsanity.com/paper/2010.05700/) (2020)
- [Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets](https://mlsanity.com/paper/1908.07898/) (2019)
- [Towards Debiasing Fact Verification Models](https://mlsanity.com/paper/1908.05267/) (2019)
- [End-to-End Bias Mitigation by Modelling Biases in Corpora](https://mlsanity.com/paper/1909.06321/) (2019)
- [Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance](https://mlsanity.com/paper/2005.00315/) (2020)
- [Fool Me Twice: Entailment from Wikipedia Gamification](https://mlsanity.com/paper/2104.04725/) (2021)
- [Adversarial attacks against Fact Extraction and VERification](https://mlsanity.com/paper/1903.05543/) (2019)

## Built on by (most cited first, 23 of 23)

- [Trustworthiness in Retrieval-Augmented Generation Systems: A Survey](https://mlsanity.com/paper/2409.10102/) (2024)
- [FactKG: Fact Verification via Reasoning on Knowledge Graphs](https://mlsanity.com/paper/2305.06590/) (2023)
- [Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasets](https://mlsanity.com/paper/2203.12942/) (2022)
- [EtiCor: Corpus for Analyzing LLMs for Etiquettes](https://mlsanity.com/paper/2310.18974/) (2023)
- [Causal Prompting: Debiasing Large Language Model Prompting based on Front-Door Adjustment](https://mlsanity.com/paper/2403.02738/) (2024)
- [A Survey on Natural Language Counterfactual Generation](https://mlsanity.com/paper/2407.03993/) (2024)
- [Improving Conversational Recommendation Systems via Bias Analysis and Language-Model-Enhanced Data Augmentation](https://mlsanity.com/paper/2310.16738/) (2023)
- [EXPLAIN, EDIT, GENERATE: Rationale-Sensitive Counterfactual Data Augmentation for Multi-hop Fact Verification](https://mlsanity.com/paper/2310.14508/) (2023)
- [Removing Non-Stationary Knowledge From Pre-Trained Language Models for Entity-Level Sentiment Classification in Finance](https://mlsanity.com/paper/2301.03136/) (2023)
- [Out-of-distribution Evidence-aware Fake News Detection via Dual Adversarial Debiasing](https://mlsanity.com/paper/2304.12888/) (2023)
- [Adversarial Attacks Against Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2509.08463/) (2025)
- [A Simple yet Effective Self-Debiasing Framework for Transformer Models](https://mlsanity.com/paper/2306.01907/) (2023)
- [Stubborn Lexical Bias in Data and Models](https://mlsanity.com/paper/2306.02190/) (2023)
- [DINER: Debiasing Aspect-based Sentiment Analysis with Multi-variable Causal Inference](https://mlsanity.com/paper/2403.01166/) (2024)
- [Robust Claim Verification Through Fact Detection](https://mlsanity.com/paper/2407.18367/) (2024)
- [IBADR: an Iterative Bias-Aware Dataset Refinement Framework for Debiasing NLU models](https://mlsanity.com/paper/2311.00292/) (2023)
- [Robust and Scalable Model Editing for Large Language Models](https://mlsanity.com/paper/2403.17431/) (2024)
- [Mitigating Shortcut Learning with InterpoLated Learning](https://mlsanity.com/paper/2507.05527/) (2025)
- [Enhancing Effectiveness and Robustness in a Low-Resource Regime via Decision-Boundary-aware Data Augmentation](https://mlsanity.com/paper/2403.15512/) (2024)
- [Multi-Evidence based Fact Verification via A Confidential Graph Neural Network](https://mlsanity.com/paper/2405.10481/) (2024)
- [SCISSOR: Mitigating Semantic Bias through Cluster-Aware Siamese Networks for Robust Classification](https://mlsanity.com/paper/2506.14587/) (2025)
- [Learning to Infer from Unlabeled Data: A Semi-supervised Learning Approach for Robust Natural Language Inference](https://mlsanity.com/paper/2211.02971/) (2022)
- [MuPlon: Multi-Path Causal Optimization for Claim Verification through Controlling Confounding](https://mlsanity.com/paper/2509.25715/) (2025)

## Abstract (arXiv)

Fact verification datasets are typically constructed using crowdsourcing techniques due to the lack of text sources with veracity labels. However, the crowdsourcing process often produces undesired biases in data that cause models to learn spurious patterns. In this paper, we propose CrossAug, a contrastive data augmentation method for debiasing fact verification models. Specifically, we employ a two-stage augmentation pipeline to generate new claims and evidences from existing samples. The generated samples are then paired cross-wise with the original pair, forming contrastive samples that facilitate the model to rely less on spurious patterns and learn more robust representations. Experimental results show that our method outperforms the previous state-of-the-art debiasing technique by 3.6% on the debiased extension of the FEVER dataset, with a total performance boost of 10.13% from the baseline. Furthermore, we evaluate our approach in data-scarce settings, where models can be more susceptible to biases due to the lack of training data. Experimental results demonstrate that our approach is also effective at debiasing in these low-resource conditions, exceeding the baseline performance on the Symmetric dataset with just 1% of the original data.