# Exploring Unsupervised Pretraining and Sentence Structure Modelling for Winograd Schema Challenge

arXiv: https://arxiv.org/abs/1904.09705 · HTML: https://mlsanity.com/paper/1904.09705/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Apr 2019.

## Built on by (most cited first, 3 of 3)

- [The Defeat of the Winograd Schema Challenge](https://mlsanity.com/paper/2201.02387/) (2022)
- [Unsupervised Pronoun Resolution via Masked Noun-Phrase Prediction](https://mlsanity.com/paper/2105.12392/) (2021)
- [Attention-based Contrastive Learning for Winograd Schemas](https://mlsanity.com/paper/2109.05108/) (2021)

## Abstract (arXiv)

Winograd Schema Challenge (WSC) was proposed as an AI-hard problem in testing computers' intelligence on common sense representation and reasoning. This paper presents the new state-of-theart on WSC, achieving an accuracy of 71.1%. We demonstrate that the leading performance benefits from jointly modelling sentence structures, utilizing knowledge learned from cutting-edge pretraining models, and performing fine-tuning. We conduct detailed analyses, showing that fine-tuning is critical for achieving the performance, but it helps more on the simpler associative problems. Modelling sentence dependency structures, however, consistently helps on the harder non-associative subset of WSC. Analysis also shows that larger fine-tuning datasets yield better performances, suggesting the potential benefit of future work on annotating more Winograd schema sentences.