# Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets

arXiv: https://arxiv.org/abs/1904.02668 · HTML: https://mlsanity.com/paper/1904.02668/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Apr 2019.

## Built on by (most cited first, 30 of 42)

- [Dynabench: Rethinking Benchmarking in NLP](https://mlsanity.com/paper/2104.14337/) (2021)
- [Measure and Improve Robustness in NLP Models: A Survey](https://mlsanity.com/paper/2112.08313/) (2021)
- [Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?](https://mlsanity.com/paper/2106.01465/) (2021)
- [Automatic Generation of Contrast Sets from Scene Graphs: Probing the Compositional Consistency of GQA](https://mlsanity.com/paper/2103.09591/) (2021)
- [Avoiding Inference Heuristics in Few-shot Prompt-based Finetuning](https://mlsanity.com/paper/2109.04144/) (2021)
- [Exploring Transitivity in Neural NLI Models through Veridicality](https://mlsanity.com/paper/2101.10713/) (2021)
- [Thinking Aloud: Dynamic Context Generation Improves Zero-Shot Reasoning Performance of GPT-2](https://mlsanity.com/paper/2103.13033/) (2021)
- [Do We Need Language-Specific Fact-Checking Models? The Case of Chinese](https://mlsanity.com/paper/2401.15498/) (2024)
- [Less Learn Shortcut: Analyzing and Mitigating Learning of Spurious Feature-Label Correlation](https://mlsanity.com/paper/2205.12593/) (2022)
- [Is My Model Using The Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoning](https://mlsanity.com/paper/2108.00578/) (2021)
- [Transparency Helps Reveal When Language Models Learn Meaning](https://mlsanity.com/paper/2210.07468/) (2022)
- [SyGNS: A Systematic Generalization Testbed Based on Natural Language Semantics](https://mlsanity.com/paper/2106.01077/) (2021)
- [Dyna-bAbI: unlocking bAbI's potential with dynamic synthetic benchmarking](https://mlsanity.com/paper/2112.00086/) (2021)
- [Assessing Out-of-Domain Language Model Performance from Few Examples](https://mlsanity.com/paper/2210.06725/) (2022)
- [Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Sets](https://mlsanity.com/paper/2311.08662/) (2023)
- [Don't Think Twice! Over-Reasoning Impairs Confidence Calibration](https://mlsanity.com/paper/2508.15050/) (2025)
- [Supporting Context Monotonicity Abstractions in Neural NLI Models](https://mlsanity.com/paper/2105.08008/) (2021)
- [Fighting Bias with Bias: Promoting Model Robustness by Amplifying Dataset Biases](https://mlsanity.com/paper/2305.18917/) (2023)
- [A Survey on Measuring and Mitigating Reasoning Shortcuts in Machine Reading Comprehension](https://mlsanity.com/paper/2209.01824/) (2022)
- [Backdoor Learning for NLP: Recent Advances, Challenges, and Future Research Directions](https://mlsanity.com/paper/2302.06801/) (2023)
- [Is "My Favorite New Movie" My Favorite Movie? Probing the Understanding of Recursive Noun Phrases](https://mlsanity.com/paper/2112.08326/) (2021)
- [Curriculum: A Broad-Coverage Benchmark for Linguistic Phenomena in Natural Language Understanding](https://mlsanity.com/paper/2204.06283/) (2022)
- [Possible Stories: Evaluating Situated Commonsense Reasoning under Multiple Possible Scenarios](https://mlsanity.com/paper/2209.07760/) (2022)
- [Enhancing Tabular Reasoning with Pattern Exploiting Training](https://mlsanity.com/paper/2210.12259/) (2022)
- [Misspellings in Natural Language Processing: A survey](https://mlsanity.com/paper/2501.16836/) (2025)
- [BERT Has Uncommon Sense: Similarity Ranking for Word Sense BERTology](https://mlsanity.com/paper/2109.09780/) (2021)
- [A Proposal to Study "Is High Quality Data All We Need?"](https://mlsanity.com/paper/2203.06404/) (2022)
- [Checking HateCheck: a cross-functional analysis of behaviour-aware learning for hate speech detection](https://mlsanity.com/paper/2204.04042/) (2022)
- [Cross-functional Analysis of Generalisation in Behavioural Learning](https://mlsanity.com/paper/2305.12951/) (2023)
- [Specification Overfitting in Artificial Intelligence](https://mlsanity.com/paper/2403.08425/) (2024)

## Abstract (arXiv)

Several datasets have recently been constructed to expose brittleness in models trained on existing benchmarks. While model performance on these challenge datasets is significantly lower compared to the original benchmark, it is unclear what particular weaknesses they reveal. For example, a challenge dataset may be difficult because it targets phenomena that current models cannot capture, or because it simply exploits blind spots in a model's specific training set. We introduce inoculation by fine-tuning, a new analysis method for studying challenge datasets by exposing models (the metaphorical patient) to a small amount of data from the challenge dataset (a metaphorical pathogen) and assessing how well they can adapt. We apply our method to analyze the NLI "stress tests" (Naik et al., 2018) and the Adversarial SQuAD dataset (Jia and Liang, 2017). We show that after slight exposure, some of these datasets are no longer challenging, while others remain difficult. Our results indicate that failures on challenge datasets may lead to very different conclusions about models, training datasets, and the challenge datasets themselves.