mlsanityData to 7 Oct 2026

A Surprisingly Robust Trick for Winograd Schema Challenge

arXiv 1905.06290 ↗First posted 15 May 2019cs.CLcited by 30

Not on the map. Why?

Abstract (arXiv)

The Winograd Schema Challenge (WSC) dataset WSC273 and its inference counterpart WNLI are popular benchmarks for natural language understanding and commonsense reasoning. In this paper, we show that the performance of three language models on WSC273 strongly improves when fine-tuned on a similar pronoun disambiguation problem dataset (denoted WSCR). We additionally generate a large unsupervised WSC-like dataset. By fine-tuning the BERT language model both on the introduced and on the WSCR dataset, we achieve overall accuracies of 72.5% and 74.7% on WSC273 and WNLI, improving the previous state-of-the-art solutions by 8.8% and 9.6%, respectively. Furthermore, our fine-tuned models are also consistently more robust on the "complex" subsets of WSC273, introduced by Trichelair et al. (2018).

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 14; 2022: 5; 2023: 5; 2024: 4; 2025: 2.

30 papers.

Sort:

By area:

  1. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  2. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
  3. 2021Merging Models with Fisher-Weighted Averagingcited by 613
  4. 2021True Few-Shot Learning with Language Modelscited by 343
  5. 2021It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoningcited by 69
  6. 2022Event knowledge in large language models: the gap between the impossible and the unlikelycited by 44
  7. 2023Trained on 100 million words and still in shape: BERT meets British National Corpuscited by 39
  8. 2021Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schemacited by 32
  9. 2022The Defeat of the Winograd Schema Challengecited by 25
  10. 2021Modeling Event Plausibility with Consistent Conceptual Abstractioncited by 20
  11. 2024Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystackcited by 19
  12. 2021From Discourse to Narrative: Knowledge Projection for Event Relation Extractioncited by 14
18 more
  1. 2021A Semantic-based Method for Unsupervised Commonsense Question Answeringcited by 10
  2. 2021Pseudo Zero Pronoun Resolution Improves Zero Anaphora Resolutioncited by 4
  3. 2021Towards Zero-shot Commonsense Reasoning with Self-supervised Refinement of Language Modelscited by 4
  4. 2023Thrust: Adaptively Propels Large Language Models with External Knowledgecited by 3
  5. 2024Picturing Ambiguity: A Visual Twist on the Winograd Schema Challengecited by 3
  6. 2021Unsupervised Pronoun Resolution via Masked Noun-Phrase Predictioncited by 2
  7. 2021Enhancing Natural Language Representation with Large-Scale Out-of-Domain Commonsensecited by 2
  8. 2022On Reality and the Limits of Language Data: Aligning LLMs with Human Normscited by 2
  9. 2024EvoGrad: A Dynamic Take on the Winograd Schema Challenge with Human Adversariescited by 2
  10. 2025NeuralDB: Scaling Knowledge Editing in LLMs to 100,000 Facts with Neural KV Databasecited by 2
  11. 2021Attention-based Contrastive Learning for Winograd Schemascited by 1
  12. 2022Brain-inspired Graph Spiking Neural Networks for Commonsense Knowledge Representation and Reasoningcited by 1
  13. 2022UU-Tax at SemEval-2022 Task 3: Improving the generalizability of language models for taxonomy classification through data augmentationcited by 1
  14. 2024Solving the Challenge Set without Solving the Task: On Winograd Schemas as a Test of Pronominal Coreference Resolutioncited by 1
  15. 2021OCHADAI-KYOTO at SemEval-2021 Task 1: Enhancing Model Generalization and Robustness for Lexical Complexity Predictioncited by 0
  16. 2023PronounFlow: A Hybrid Approach for Calibrating Pronouns in Sentencescited by 0
  17. 2023Experience and Prediction: A Metric of Hardness for a Novel Litmus Testcited by 0
  18. 2025Estonian WinoGrande Dataset: Comparative Analysis of LLM Performance on Human and Machine Translationcited by 0