mlsanityData to 7 Oct 2026

SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference

arXiv 1808.05326 ↗First posted 16 Aug 2018cs.CLcited by 333

Not on the map. Why?

Cited by 44 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Given a partial description like "she opened the hood of the car," humans can reason about the situation and anticipate what might come next ("then, she examined the engine"). In this paper, we introduce the task of grounded commonsense inference, unifying natural language inference and commonsense reasoning. We present SWAG, a new dataset with 113k multiple choice questions about a rich spectrum of grounded situations. To address the recurring challenges of the annotation artifacts and human biases found in many existing datasets, we propose Adversarial Filtering (AF), a novel procedure that constructs a de-biased dataset by iteratively training an ensemble of stylistic classifiers, and using them to filter the data. To account for the aggressive adversarial filtering, we use state-of-the-art language models to massively oversample a diverse set of potential counterfactuals. Empirical results demonstrate that while humans can solve the resulting inference problems with high accuracy (88%), various competitive models struggle on our task. We provide comprehensive analysis that indicates significant opportunities for future research.

Key facts. Cited by 333 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 91; 2022: 87; 2023: 59; 2024: 45; 2025: 36; 2026: 15.

333 papers.

Sort:

By area:

  1. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  2. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  3. 2021MetaICL: Learning to Learn In Contextcited by 450
  4. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 411
  5. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  6. 2022RealTime QA: What's the Answer Right Now?cited by 239
  7. 2023SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionalitycited by 238
  8. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 231
  9. 2021What Will it Take to Fix Benchmarking in Natural Language Understanding?cited by 200
  10. 2022Leveraging Large Language Models for Multiple Choice Question Answeringcited by 199
  11. 2021Muppet: Massive Multi-task Representations with Pre-Finetuningcited by 192
  12. 2022Language model compression with weighted low-rank factorizationcited by 192
38 more
  1. 2022Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Modelscited by 179
  2. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  3. 2023Active Prompting with Chain-of-Thought for Large Language Modelscited by 154
  4. 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
  5. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  6. 2021A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generationcited by 137
  7. 2021BERT: A Review of Applications in Natural Language Processing and Understandingcited by 109
  8. 2022CommonsenseQA 2.0: Exposing the Limits of AI through Gamificationcited by 108
  9. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  10. 2022Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysiscited by 104
  11. 2021UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmarkcited by 98
  12. 2022What Artificial Neural Networks Can Tell Us About Human Language Acquisitioncited by 97
  13. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  14. 2022On the Paradox of Learning to Reason from Datacited by 93
  15. 2021Do Long-Range Language Models Actually Use Long-Range Context?cited by 92
  16. 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
  17. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
  18. 2021CREAK: A Dataset for Commonsense Reasoning over Entity Knowledgecited by 88
  19. 2022Shortcut Learning of Large Language Models in Natural Language Understandingcited by 83
  20. 2023Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Imagescited by 79
  21. 2021Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Modelscited by 78
  22. 2022Efficient Methods for Natural Language Processing: A Surveycited by 78
  23. 2021Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Modelscited by 74
  24. 2022Revisiting Over-smoothing in BERT from the Perspective of Graphcited by 71
  25. 2022Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examplescited by 65
  26. 2021Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoningcited by 64
  27. 2021What to Pre-Train on? Efficient Intermediate Task Selectioncited by 62
  28. 2021Can Generative Pre-trained Language Models Serve as Knowledge Bases for Closed-book QA?cited by 62
  29. 2023Black Box Adversarial Prompting for Foundation Modelscited by 62
  30. 2023TeleQnA: A Benchmark Dataset to Assess Large Language Models Telecommunications Knowledgecited by 62
  31. 2023Natural Language Reasoning, A Surveycited by 61
  32. 2021RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledgecited by 60
  33. 2021Broaden the Vision: Geo-Diverse Visual Commonsense Reasoningcited by 59
  34. 2021ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoningcited by 58
  35. 2023Training Transformers with 4-bit Integerscited by 58
  36. 2022Housekeep: Tidying Virtual Households using Commonsense Reasoningcited by 57
  37. 2021A Systematic Investigation of Commonsense Knowledge in Large Language Modelscited by 56
  38. 2021Comparing Test Sets with Item Response Theorycited by 55

50 most cited of 333 All 333 (JSON)