# A Surprisingly Robust Trick for Winograd Schema Challenge

arXiv: https://arxiv.org/abs/1905.06290 · HTML: https://mlsanity.com/paper/1905.06290/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 May 2019.

## Built on by (most cited first, 30 of 30)

- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch](https://mlsanity.com/paper/2311.03099/) (2023)
- [Merging Models with Fisher-Weighted Averaging](https://mlsanity.com/paper/2111.09832/) (2021)
- [True Few-Shot Learning with Language Models](https://mlsanity.com/paper/2105.11447/) (2021)
- [It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning](https://mlsanity.com/paper/2106.12066/) (2021)
- [Event knowledge in large language models: the gap between the impossible and the unlikely](https://mlsanity.com/paper/2212.01488/) (2022)
- [Trained on 100 million words and still in shape: BERT meets British National Corpus](https://mlsanity.com/paper/2303.09859/) (2023)
- [Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema](https://mlsanity.com/paper/2104.08161/) (2021)
- [The Defeat of the Winograd Schema Challenge](https://mlsanity.com/paper/2201.02387/) (2022)
- [Modeling Event Plausibility with Consistent Conceptual Abstraction](https://mlsanity.com/paper/2104.10247/) (2021)
- [Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack](https://mlsanity.com/paper/2407.16695/) (2024)
- [From Discourse to Narrative: Knowledge Projection for Event Relation Extraction](https://mlsanity.com/paper/2106.08629/) (2021)
- [A Semantic-based Method for Unsupervised Commonsense Question Answering](https://mlsanity.com/paper/2105.14781/) (2021)
- [Pseudo Zero Pronoun Resolution Improves Zero Anaphora Resolution](https://mlsanity.com/paper/2104.07425/) (2021)
- [Towards Zero-shot Commonsense Reasoning with Self-supervised Refinement of Language Models](https://mlsanity.com/paper/2109.05105/) (2021)
- [Thrust: Adaptively Propels Large Language Models with External Knowledge](https://mlsanity.com/paper/2307.10442/) (2023)
- [Picturing Ambiguity: A Visual Twist on the Winograd Schema Challenge](https://mlsanity.com/paper/2405.16277/) (2024)
- [Unsupervised Pronoun Resolution via Masked Noun-Phrase Prediction](https://mlsanity.com/paper/2105.12392/) (2021)
- [Enhancing Natural Language Representation with Large-Scale Out-of-Domain Commonsense](https://mlsanity.com/paper/2109.02572/) (2021)
- [On Reality and the Limits of Language Data: Aligning LLMs with Human Norms](https://mlsanity.com/paper/2208.11981/) (2022)
- [EvoGrad: A Dynamic Take on the Winograd Schema Challenge with Human Adversaries](https://mlsanity.com/paper/2402.13372/) (2024)
- [NeuralDB: Scaling Knowledge Editing in LLMs to 100,000 Facts with Neural KV Database](https://mlsanity.com/paper/2507.18028/) (2025)
- [Attention-based Contrastive Learning for Winograd Schemas](https://mlsanity.com/paper/2109.05108/) (2021)
- [Brain-inspired Graph Spiking Neural Networks for Commonsense Knowledge Representation and Reasoning](https://mlsanity.com/paper/2207.05561/) (2022)
- [UU-Tax at SemEval-2022 Task 3: Improving the generalizability of language models for taxonomy classification through data augmentation](https://mlsanity.com/paper/2210.03378/) (2022)
- [Solving the Challenge Set without Solving the Task: On Winograd Schemas as a Test of Pronominal Coreference Resolution](https://mlsanity.com/paper/2410.09448/) (2024)
- [OCHADAI-KYOTO at SemEval-2021 Task 1: Enhancing Model Generalization and Robustness for Lexical Complexity Prediction](https://mlsanity.com/paper/2105.05535/) (2021)
- [PronounFlow: A Hybrid Approach for Calibrating Pronouns in Sentences](https://mlsanity.com/paper/2308.15235/) (2023)
- [Experience and Prediction: A Metric of Hardness for a Novel Litmus Test](https://mlsanity.com/paper/2309.02534/) (2023)
- [Estonian WinoGrande Dataset: Comparative Analysis of LLM Performance on Human and Machine Translation](https://mlsanity.com/paper/2511.17290/) (2025)

## Abstract (arXiv)

The Winograd Schema Challenge (WSC) dataset WSC273 and its inference counterpart WNLI are popular benchmarks for natural language understanding and commonsense reasoning. In this paper, we show that the performance of three language models on WSC273 strongly improves when fine-tuned on a similar pronoun disambiguation problem dataset (denoted WSCR). We additionally generate a large unsupervised WSC-like dataset. By fine-tuning the BERT language model both on the introduced and on the WSCR dataset, we achieve overall accuracies of 72.5% and 74.7% on WSC273 and WNLI, improving the previous state-of-the-art solutions by 8.8% and 9.6%, respectively. Furthermore, our fine-tuned models are also consistently more robust on the "complex" subsets of WSC273, introduced by Trichelair et al. (2018).