# Unpacking the Resilience of SNLI Contradiction Examples to Attacks

arXiv: https://arxiv.org/abs/2412.11172 · HTML: https://mlsanity.com/paper/2412.11172/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 12 references that are in the map.
- First posted on arXiv 15 Dec 2024.

## Builds on (most cited first, 5 of 5)

- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Annotation Artifacts in Natural Language Inference Data](https://mlsanity.com/paper/1803.02324/) (2018)
- [Hypothesis Only Baselines in Natural Language Inference](https://mlsanity.com/paper/1805.01042/) (2018)
- [Enhanced LSTM for Natural Language Inference](https://mlsanity.com/paper/1609.06038/) (2016)
- [Universal Adversarial Attacks with Natural Triggers for Text Classification](https://mlsanity.com/paper/2005.00174/) (2020)

## Abstract (arXiv)

Pre-trained models excel on NLI benchmarks like SNLI and MultiNLI, but their true language understanding remains uncertain. Models trained only on hypotheses and labels achieve high accuracy, indicating reliance on dataset biases and spurious correlations. To explore this issue, we applied the Universal Adversarial Attack to examine the model's vulnerabilities. Our analysis revealed substantial drops in accuracy for the entailment and neutral classes, whereas the contradiction class exhibited a smaller decline. Fine-tuning the model on an augmented dataset with adversarial examples restored its performance to near-baseline levels for both the standard and challenge sets. Our findings highlight the value of adversarial triggers in identifying spurious correlations and improving robustness while providing insights into the resilience of the contradiction class to adversarial attacks.