mlsanityData to 7 Oct 2026

HellaSwag: Can a Machine Really Finish Your Sentence?

arXiv 1905.07830 ↗First posted 19 May 2019cs.CLcited by 4,018

Not on the map. Why?

Cited by 280 in Mixture-of-experts, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent work by Zellers et al. (2018) introduced a new task of commonsense natural language inference: given an event description such as "A woman sits at a piano," a machine must select the most likely followup: "She sets her fingers on the keys." With the introduction of BERT, near human-level performance was reached. Does this mean that machines can perform human level commonsense inference? In this paper, we show that commonsense inference still proves difficult for even state-of-the-art models, by presenting HellaSwag, a new challenge dataset. Though its questions are trivial for humans (>95% accuracy), state-of-the-art models struggle (<48%). We achieve this via Adversarial Filtering (AF), a data collection paradigm wherein a series of discriminators iteratively select an adversarial set of machine-generated wrong answers. AF proves to be surprisingly robust. The key insight is to scale up the length and complexity of the dataset examples towards a critical 'Goldilocks' zone wherein generated text is ridiculous to humans, yet often misclassified by state-of-the-art models. Our construction of HellaSwag, and its resulting difficulty, sheds light on the inner workings of deep pretrained models. More broadly, it suggests a new path forward for NLP research, in which benchmarks co-evolve with the evolving state-of-the-art in an adversarial way, so as to present ever-harder challenges.

Key facts. Cited by 4,018 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 64; 2022: 104; 2023: 304; 2024: 1060; 2025: 1303; 2026: 1183.

4,018 papers.

Sort:

By area:

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  10. 2023Qwen Technical Reportcited by 3,273
  11. 2023Mistral 7Bcited by 3,065
  12. 2022Training Compute-Optimal Large Language Modelscited by 2,927
38 more
  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  4. 2024Qwen2 Technical Reportcited by 2,076
  5. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  6. 2024Mixtral of Expertscited by 1,785
  7. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  8. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  9. 2025Gemma 3 Technical Reportcited by 1,686
  10. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  11. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  12. 2023PaLM 2 Technical Reportcited by 1,154
  13. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  14. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  15. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  16. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  17. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  18. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  19. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  20. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  21. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  22. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  23. 2023BloombergGPT: A Large Language Model for Financecited by 923
  24. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  25. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  26. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  27. 2021Measuring Coding Challenge Competence With APPScited by 879
  28. 2025Large Language Diffusion Modelscited by 872
  29. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  30. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  31. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  32. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  33. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  34. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  35. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  36. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  37. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  38. 2024Yi: Open Foundation Models by 01.AIcited by 706

50 most cited of 4,018 All 4,018 (JSON)