mlsanityData to 7 Oct 2026

How Reasonable are Common-Sense Reasoning Tasks: A Case-Study on the Winograd Schema Challenge and SWAG

arXiv 1811.01778 ↗First posted 5 Nov 2018cs.LG cs.AI cs.CLcited by 18

Not on the map. Why?

Abstract (arXiv)

Recent studies have significantly improved the state-of-the-art on common-sense reasoning (CSR) benchmarks like the Winograd Schema Challenge (WSC) and SWAG. The question we ask in this paper is whether improved performance on these benchmarks represents genuine progress towards common-sense-enabled systems. We make case studies of both benchmarks and design protocols that clarify and qualify the results of previous work by analyzing threats to the validity of previous experimental designs. Our protocols account for several properties prevalent in common-sense benchmarks including size limitations, structural regularities, and variable instance difficulty.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 3; 2023: 4; 2024: 3; 2025: 2.

18 papers.

Sort:

By area:

  1. 2022Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contestcited by 98
  2. 2021A Systematic Investigation of Commonsense Knowledge in Large Language Modelscited by 56
  3. 2021Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schemacited by 32
  4. 2023NLI4CT: Multi-Evidence Natural Language Inference for Clinical Trial Reportscited by 18
  5. 2024Robust Pronoun Fidelity with English LLMs: Are they Reasoning, Repeating, or Just Biased?cited by 17
  6. 2022Are Prompt-based Models Clueless?cited by 11
  7. 2023Toward best research practices in AI Psychologycited by 8
  8. 2023Debiasing should be Good and Bad: Measuring the Consistency of Debiasing Techniques in Language Modelscited by 7
  9. 2023Causal interventions expose implicit situation models for commonsense language understandingcited by 7
  10. 2025Who's Laughing Now? An Overview of Computational Humour Generation and Explanationcited by 5
  11. 2021Towards Zero-shot Commonsense Reasoning with Self-supervised Refinement of Language Modelscited by 4
  12. 2022On Reality and the Limits of Language Data: Aligning LLMs with Human Normscited by 2
6 more
  1. 2024EvoGrad: A Dynamic Take on the Winograd Schema Challenge with Human Adversariescited by 2
  2. 2024Ranking LLMs by compressioncited by 2
  3. 2025WinoWhat: A Parallel Corpus of Paraphrased WinoGrande Sentences with Common Sense Categorizationcited by 2
  4. 2021Attention-based Contrastive Learning for Winograd Schemascited by 1
  5. 2021Problems and Countermeasures in Natural Language Processing Evaluationcited by 0
  6. 2021Go Beyond Plain Fine-tuning: Improving Pretrained Models for Social Commonsensecited by 0