mlsanityData to 7 Oct 2026

Adversarial NLI: A New Benchmark for Natural Language Understanding

arXiv 1910.14599 ↗First posted 31 Oct 2019cs.CL cs.LGcited by 798

Not on the map. Why?

Cited by 54 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce a new large-scale NLI benchmark dataset, collected via an iterative, adversarial human-and-model-in-the-loop procedure. We show that training models on this new dataset leads to state-of-the-art performance on a variety of popular NLI benchmarks, while posing a more difficult challenge with its new test set. Our analysis sheds light on the shortcomings of current state-of-the-art models, and shows that non-expert annotators are successful at finding their weaknesses. The data collection method can be applied in a never-ending learning scenario, becoming a moving target for NLU, rather than a static benchmark that will quickly saturate.

Key facts. Cited by 798 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 112; 2022: 132; 2023: 186; 2024: 163; 2025: 123; 2026: 82.

798 papers.

Sort:

By area:

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  5. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  6. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  7. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  8. 2023PaLM 2 Technical Reportcited by 1,154
  9. 2022Red Teaming Language Models with Language Modelscited by 983
  10. 2023BloombergGPT: A Large Language Model for Financecited by 923
  11. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  12. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
38 more
  1. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  2. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  3. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  4. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  5. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  6. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  7. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  8. 2022Large Language Models Can Self-Improvecited by 656
  9. 2023A Comprehensive Overview of Large Language Modelscited by 639
  10. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  11. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
  12. 2025Humanity's Last Examcited by 467
  13. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  14. 2021MetaICL: Learning to Learn In Contextcited by 450
  15. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  16. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  17. 2023Symbolic Discovery of Optimization Algorithmscited by 423
  18. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  19. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 402
  20. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  21. 2022ST-MoE: Designing Stable and Transferable Sparse Expert Modelscited by 391
  22. 2023Scaling Data-Constrained Language Modelscited by 391
  23. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  24. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  25. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
  26. 2024tinyBenchmarks: evaluating LLMs with fewer examplescited by 295
  27. 2022UL2: Unifying Language Learning Paradigmscited by 292
  28. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 284
  29. 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 281
  30. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 251
  31. 2021SPoT: Better Frozen Model Adaptation through Soft Prompt Transfercited by 235
  32. 2021QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarizationcited by 233
  33. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 227
  34. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  35. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
  36. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 218
  37. 2023On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspectivecited by 208
  38. 2021QuALITY: Question Answering with Long Input Texts, Yes!cited by 204

50 most cited of 798 All 798 (JSON)