mlsanityData to 7 Oct 2026

ANLIzing the Adversarial Natural Language Inference Dataset

arXiv 2010.12729 ↗First posted 24 Oct 2020cs.CLcited by 36

Not on the map. Why?

Abstract (arXiv)

We perform an in-depth error analysis of Adversarial NLI (ANLI), a recently introduced large-scale human-and-model-in-the-loop natural language inference dataset collected over multiple rounds. We propose a fine-grained annotation scheme of the different aspects of inference that are responsible for the gold classification labels, and use it to hand-code all three of the ANLI development sets. We use these annotations to answer a variety of interesting questions: which inference types are most common, which models have the highest performance on each reasoning type, and which types are the most challenging for state of-the-art models? We hope that our annotations will enable more fine-grained evaluation of models trained on ANLI, provide us with a deeper understanding of where models fail and succeed, and help us determine how to train better models in future.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 13; 2023: 6; 2024: 3; 2025: 5; 2026: 1.

36 papers.

Sort:

By area:

  1. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  2. 2022SafeText: A Benchmark for Exploring Physical Safety in Language Modelscited by 41
  3. 2021The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Failcited by 38
  4. 2021On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Studycited by 30
  5. 2021Does Putting a Linguist in the Loop Improve NLU Data Collection?cited by 29
  6. 2022Investigating Reasons for Disagreement in Natural Language Inferencecited by 29
  7. 2024The Factorization Curse: Which Tokens You Predict Underlie the Reversal Curse and Morecited by 25
  8. 2021Analyzing Dynamic Adversarial Training Data in the Limitcited by 24
  9. 2021IndoNLI: A Natural Language Inference Dataset for Indonesiancited by 24
  10. 2024Long-form evaluation of model editingcited by 23
  11. 2022A Survey of Adversarial Defences and Robustness in NLPcited by 22
  12. 2022ALERT: Adapting Language Models to Reasoning Taskscited by 17
24 more
  1. 2022Dynatask: A Framework for Creating Dynamic AI Benchmark Taskscited by 16
  2. 2025ARGUS: Hallucination and Omission Evaluation in Video-LLMscited by 16
  3. 2021He Thinks He Knows Better than the Doctors: BERT for Event Factuality Fails on Pragmaticscited by 12
  4. 2021Adversarially Constructed Evaluation Sets Are More Challenging, but May Not Be Faircited by 12
  5. 2022What Makes Reading Comprehension Questions Difficult?cited by 11
  6. 2022Random-LTD: Random and Layerwise Token Dropping Brings Efficient Training for Large-scale Transformerscited by 11
  7. 2022JamPatoisNLI: A Jamaican Patois Natural Language Inference Datasetcited by 11
  8. 2025Always Tell Me The Odds: Fine-grained Conditional Probability Estimationcited by 9
  9. 2022Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarkscited by 8
  10. 2022longhorns at DADC 2022: How many linguists does it take to fool a Question Answering model? A systematic approach to adversarial attackscited by 8
  11. 2023Training Large Language Models Efficiently with Sparsity and Dataflowcited by 7
  12. 2025LegalWiz: A Multi-Agent Generation Framework for Contradiction Detection in Legal Documentscited by 7
  13. 2021Knowledge-Enhanced Evidence Retrieval for Counterargument Generationcited by 6
  14. 2023MISMATCH: Fine-grained Evaluation of Machine-generated Text with Mismatch Error Typescited by 4
  15. 2022Analyzing Multi-Task Learning for Abstractive Text Summarizationcited by 3
  16. 2022Looking at the Overlooked: An Analysis on the Word-Overlap Bias in Natural Language Inferencecited by 3
  17. 2023Who's the Best Detective? LLMs vs. MLs in Detecting Incoherent Fourth Grade Math Answerscited by 3
  18. 2025Giving AI Personalities Leads to More Human-Like Reasoningcited by 3
  19. 2023What does the Failure to Reason with "Respectively" in Zero/Few-Shot Settings Tell Us about Language Models?cited by 2
  20. 2023Prefer to Classify: Improving Text Classifiers via Auxiliary Preference Learningcited by 2
  21. 2023Contrastive Learning for Inference in Dialoguecited by 0
  22. 2024Analysis of Socially Unacceptable Discourse with Zero-shot Learningcited by 0
  23. 2025A Systematic Analysis of Base Model Choice for Reward Modelingcited by 0
  24. 2026A Systematic Investigation of RL-Jailbreaking in LLMscited by 0