# A Review of Winograd Schema Challenge Datasets and Approaches

arXiv: https://arxiv.org/abs/2004.13831 · HTML: https://mlsanity.com/paper/2004.13831/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2020.

## Built on by (most cited first, 27 of 27)

- [Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality](https://mlsanity.com/paper/2204.03162/) (2022)
- [Meaning without reference in large language models](https://mlsanity.com/paper/2208.02957/) (2022)
- [Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema](https://mlsanity.com/paper/2104.08161/) (2021)
- [WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models](https://mlsanity.com/paper/2207.12576/) (2022)
- [Sabi'a: Portuguese Large Language Models](https://mlsanity.com/paper/2304.07880/) (2023)
- [BLUEX: A benchmark based on Brazilian Leading Universities Entrance eXams](https://mlsanity.com/paper/2307.05410/) (2023)
- [Efficiency optimization of large-scale language models based on deep learning in natural language processing tasks](https://mlsanity.com/paper/2405.11704/) (2024)
- [Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization](https://mlsanity.com/paper/2405.15861/) (2024)
- [When Attention Collapses: How Degenerate Layers in LLMs Enable Smaller, Stronger Models](https://mlsanity.com/paper/2404.08634/) (2024)
- [Baby's CoThought: Leveraging Large Language Models for Enhanced Reasoning in Compact Models](https://mlsanity.com/paper/2308.01684/) (2023)
- [C3: A Bilingual Benchmark for Spoken Dialogue Models Exploring Challenges in Complex Conversations](https://mlsanity.com/paper/2507.22968/) (2025)
- [Neuron-based Multifractal Analysis of Neuron Interaction Dynamics in Large Models](https://mlsanity.com/paper/2402.09099/) (2024)
- [The Case for Claim Difficulty Assessment in Automatic Fact Checking](https://mlsanity.com/paper/2109.09689/) (2021)
- [Saliency-driven Dynamic Token Pruning for Large Language Models](https://mlsanity.com/paper/2504.04514/) (2025)
- [Multilingual Test-Time Scaling via Initial Thought Transfer](https://mlsanity.com/paper/2505.15508/) (2025)
- [Towards Zero-shot Commonsense Reasoning with Self-supervised Refinement of Language Models](https://mlsanity.com/paper/2109.05105/) (2021)
- [Towards Better Few-Shot and Finetuning Performance with Forgetful Causal Language Models](https://mlsanity.com/paper/2210.13432/) (2022)
- [Analyzing Multi-Task Learning for Abstractive Text Summarization](https://mlsanity.com/paper/2210.14606/) (2022)
- [HOSL: Hybrid-Order Split Learning for Memory-Constrained Edge Training](https://mlsanity.com/paper/2601.10940/) (2026)
- [Unsupervised Pronoun Resolution via Masked Noun-Phrase Prediction](https://mlsanity.com/paper/2105.12392/) (2021)
- [On Reality and the Limits of Language Data: Aligning LLMs with Human Norms](https://mlsanity.com/paper/2208.11981/) (2022)
- [MI-PRUN: Optimize Large Language Model Pruning via Mutual Information](https://mlsanity.com/paper/2601.07212/) (2026)
- [Revisiting the Prepositional-Phrase Attachment Problem Using Explicit Commonsense Knowledge](https://mlsanity.com/paper/2102.00924/) (2021)
- [Attention-based Contrastive Learning for Winograd Schemas](https://mlsanity.com/paper/2109.05108/) (2021)
- [Evaluate Confidence Instead of Perplexity for Zero-shot Commonsense Reasoning](https://mlsanity.com/paper/2208.11007/) (2022)
- [K"unstliche Intelligenz, quo vadis?](https://mlsanity.com/paper/2104.11067/) (2021)
- [Experience and Prediction: A Metric of Hardness for a Novel Litmus Test](https://mlsanity.com/paper/2309.02534/) (2023)

## Abstract (arXiv)

The Winograd Schema Challenge is both a commonsense reasoning and natural language understanding challenge, introduced as an alternative to the Turing test. A Winograd schema is a pair of sentences differing in one or two words with a highly ambiguous pronoun, resolved differently in the two sentences, that appears to require commonsense knowledge to be resolved correctly. The examples were designed to be easily solvable by humans but difficult for machines, in principle requiring a deep understanding of the content of the text and the situation it describes. This paper reviews existing Winograd Schema Challenge benchmark datasets and approaches that have been published since its introduction.