# Evaluating Models' Local Decision Boundaries via Contrast Sets

arXiv: https://arxiv.org/abs/2004.02709 · HTML: https://mlsanity.com/paper/2004.02709/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2020.

## Built on by (most cited first, 24 of 24)

- [MuSiQue: Multihop Questions via Single-hop Question Composition](https://mlsanity.com/paper/2108.00573/) (2021)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [Robustness Gym: Unifying the NLP Evaluation Landscape](https://mlsanity.com/paper/2101.04840/) (2021)
- [Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization](https://mlsanity.com/paper/2105.05612/) (2021)
- [Complex Knowledge Base Question Answering: A Survey](https://mlsanity.com/paper/2108.06688/) (2021)
- [Can NLI Models Verify QA Systems' Predictions?](https://mlsanity.com/paper/2104.08731/) (2021)
- [An Investigation of the (In)effectiveness of Counterfactually Augmented Data](https://mlsanity.com/paper/2107.00753/) (2021)
- [Evaluating Gender Bias in Natural Language Inference](https://mlsanity.com/paper/2105.05541/) (2021)
- [Predicting is not Understanding: Recognizing and Addressing Underspecification in Machine Learning](https://mlsanity.com/paper/2207.02598/) (2022)
- [Position: Key Claims in LLM Research Have a Long Tail of Footnotes](https://mlsanity.com/paper/2308.07120/) (2023)
- [RAGEN-2: Reasoning Collapse in Agentic RL](https://mlsanity.com/paper/2604.06268/) (2026)
- [GLUECons: A Generic Benchmark for Learning Under Constraints](https://mlsanity.com/paper/2302.10914/) (2023)
- [Cyberbullying Classifiers are Sensitive to Model-Agnostic Perturbations](https://mlsanity.com/paper/2201.06384/) (2022)
- [Building Human-like Communicative Intelligence: A Grounded Perspective](https://mlsanity.com/paper/2201.02734/) (2022)
- [Quantifying the Task-Specific Information in Text-Based Classifications](https://mlsanity.com/paper/2110.08931/) (2021)
- [Question Generation for Evaluating Cross-Dataset Shifts in Multi-modal Grounding](https://mlsanity.com/paper/2201.09639/) (2022)
- [Improving Compositional Generalization in Math Word Problem Solving](https://mlsanity.com/paper/2209.01352/) (2022)
- [Model Agnostic Answer Reranking System for Adversarial Question Answering](https://mlsanity.com/paper/2102.03016/) (2021)
- [A Proposal to Study "Is High Quality Data All We Need?"](https://mlsanity.com/paper/2203.06404/) (2022)
- [Evaluating Large Language Models Using Contrast Sets: An Experimental Approach](https://mlsanity.com/paper/2404.01569/) (2024)
- [BabelJudge: Measuring LLM-as-a-Judge Reliability Across Languages and Agent Trajectories](https://mlsanity.com/paper/2606.22329/) (2026)
- [A Survey of Parameters Associated with the Quality of Benchmarks in NLP](https://mlsanity.com/paper/2210.07566/) (2022)
- [Improving Classifier Robustness through Active Generation of Pairwise Counterfactuals](https://mlsanity.com/paper/2305.13535/) (2023)
- ["Beware of deception": Detecting Half-Truth and Debunking it through Controlled Claim Editing](https://mlsanity.com/paper/2308.07973/) (2023)

## Abstract (arXiv)

Standard test sets for supervised learning evaluate in-distribution generalization. Unfortunately, when a dataset has systematic gaps (e.g., annotation artifacts), these evaluations are misleading: a model can learn simple decision rules that perform well on the test set but do not capture a dataset's intended capabilities. We propose a new annotation paradigm for NLP that helps to close systematic gaps in the test data. In particular, after a dataset is constructed, we recommend that the dataset authors manually perturb the test instances in small but meaningful ways that (typically) change the gold label, creating contrast sets. Contrast sets provide a local view of a model's decision boundary, which can be used to more accurately evaluate a model's true linguistic capabilities. We demonstrate the efficacy of contrast sets by creating them for 10 diverse NLP datasets (e.g., DROP reading comprehension, UD parsing, IMDb sentiment analysis). Although our contrast sets are not explicitly adversarial, model performance is significantly lower on them than on the original test sets---up to 25% in some cases. We release our contrast sets as new evaluation benchmarks and encourage future dataset construction efforts to follow similar annotation processes.