mlsanityData to 7 Oct 2026

Stress Test Evaluation for Natural Language Inference

arXiv 1806.00692 ↗First posted 2 Jun 2018cs.CLcited by 211

Not on the map. Why?

Abstract (arXiv)

Natural language inference (NLI) is the task of determining if a natural language hypothesis can be inferred from a given premise in a justifiable manner. NLI was proposed as a benchmark task for natural language understanding. Existing models perform well at standard datasets for NLI, achieving impressive results across different genres of text. However, the extent to which these models understand the semantic content of sentences is unclear. In this work, we propose an evaluation methodology consisting of automatically constructed "stress tests" that allow us to examine whether systems have the ability to make real inferential decisions. Our evaluation of six sentence-encoder models on these stress tests reveals strengths and weaknesses of these models with respect to challenging linguistic phenomena, and suggests important directions for future work in this area.

Key facts. Cited by 211 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 50; 2022: 51; 2023: 42; 2024: 23; 2025: 33; 2026: 12.

211 papers.

Sort:

By area:

  1. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  2. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 227
  3. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 219
  4. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 218
  5. 2021What Will it Take to Fix Benchmarking in Natural Language Understanding?cited by 200
  6. 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 177
  7. 2022mGPT: Few-Shot Learners Go Multilingualcited by 141
  8. 2021Robustness Gym: Unifying the NLP Evaluation Landscapecited by 107
  9. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  10. 2023Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debatecited by 92
  11. 2021The MultiBERTs: BERT Reproductions for Robustness Analysiscited by 89
  12. 2021Types of Out-of-Distribution Texts and How to Detect Themcited by 80
38 more
  1. 2021Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Testscited by 77
  2. 2021Contrastive Explanations for Model Interpretabilitycited by 75
  3. 2021Understanding by Understanding Not: Modeling Negation in Language Modelscited by 73
  4. 2021Tailor: Generating and Perturbing Text with Semantic Controlscited by 67
  5. 2022Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settingscited by 66
  6. 2023VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Modelscited by 66
  7. 2023Adversarial Demonstration Attacks on Large Language Modelscited by 58
  8. 2022Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictionscited by 57
  9. 2022Linear Connectivity Reveals Generalization Strategiescited by 55
  10. 2024Elements of World Knowledge (EWoK): A Cognition-Inspired Framework for Evaluating Basic World Knowledge in Language Modelscited by 55
  11. 2022Describing Differences between Text Distributions with Natural Languagecited by 54
  12. 2022Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasetscited by 54
  13. 2021Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarkingcited by 53
  14. 2022Predicting Out-of-Distribution Error with the Projection Normcited by 50
  15. 2022On the Blind Spots of Model-Based Evaluation Metrics for Text Generationcited by 48
  16. 2021Can NLI Models Verify QA Systems' Predictions?cited by 46
  17. 2021An Investigation of the (In)effectiveness of Counterfactually Augmented Datacited by 42
  18. 2023On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codexcited by 41
  19. 2022An Analysis of Negation in Natural Language Understanding Corporacited by 39
  20. 2021Combining Feature and Instance Attribution to Detect Artifactscited by 38
  21. 2022CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negationcited by 38
  22. 2021Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Levelcited by 34
  23. 2025What Prompts Don't Say: Understanding and Managing Underspecification in LLM Promptscited by 32
  24. 2023Large Language Model Instruction Following: A Survey of Progresses and Challengescited by 31
  25. 2021Evaluating Entity Disambiguation and the Role of Popularity in Retrieval-Based NLPcited by 30
  26. 2021Does Putting a Linguist in the Loop Improve NLU Data Collection?cited by 29
  27. 2022DISCO: Distilling Counterfactuals with Large Language Modelscited by 29
  28. 2021Flexible Generation of Natural Language Deductionscited by 28
  29. 2021Grounding Representation Similarity with Statistical Testingcited by 28
  30. 2021Retrieval-guided Counterfactual Generation for QAcited by 28
  31. 2021Figurative Language in Recognizing Textual Entailmentcited by 26
  32. 2024B-AVIBench: Towards Evaluating the Robustness of Large Vision-Language Model on Black-box Adversarial Visual-Instructionscited by 26
  33. 2021NoiseQA: Challenge Set Evaluation for User-Centric Question Answeringcited by 25
  34. 2024The Factorization Curse: Which Tokens You Predict Underlie the Reversal Curse and Morecited by 25
  35. 2025reWordBench: Benchmarking and Improving the Robustness of Reward Models with Transformed Inputscited by 25
  36. 2021Analyzing Dynamic Adversarial Training Data in the Limitcited by 24
  37. 2021IndoNLI: A Natural Language Inference Dataset for Indonesiancited by 24
  38. 2021Avoiding Inference Heuristics in Few-shot Prompt-based Finetuningcited by 23

50 most cited of 211 All 211 (JSON)