mlsanityData to 7 Oct 2026

Hypothesis Only Baselines in Natural Language Inference

arXiv 1805.01042 ↗First posted 2 May 2018cs.CLcited by 291

Not on the map. Why?

Abstract (arXiv)

We propose a hypothesis only baseline for diagnosing Natural Language Inference (NLI). Especially when an NLI dataset assumes inference is occurring based purely on the relationship between a context and a hypothesis, it follows that assessing entailment relations while ignoring the provided context is a degenerate solution. Yet, through experiments on ten distinct NLI datasets, we find that this approach, which we refer to as a hypothesis-only model, is able to significantly outperform a majority class baseline across a number of NLI datasets. Our analysis suggests that statistical irregularities may allow a model to perform NLI in some datasets beyond what should be achievable without access to the context.

Key facts. Cited by 291 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 90; 2022: 65; 2023: 50; 2024: 26; 2025: 29; 2026: 31.

291 papers.

Sort:

By area:

  1. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  2. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  3. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  4. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  5. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  6. 2021A Survey on Automated Fact-Checkingcited by 376
  7. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 288
  8. 2023A Long Way to Go: Investigating Length Correlations in RLHFcited by 285
  9. 2022Prompting GPT-3 To Be Reliablecited by 258
  10. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
  11. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  12. 2024Are We Done with MMLU?cited by 178
38 more
  1. 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 177
  2. 2021Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Littlecited by 176
  3. 2023Sources of Hallucination by Large Language Models on Inference Taskscited by 168
  4. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
  5. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  6. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  7. 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
  8. 2023WiCE: Real-World Entailment for Claims in Wikipediacited by 118
  9. 2021HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generationcited by 117
  10. 2021KLUE: Korean Language Understanding Evaluationcited by 113
  11. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  12. 2022Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contestcited by 98
  13. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  14. 2021Competency Problems: On Finding and Removing Artifacts in Language Datacited by 91
  15. 2021X-FACT: A New Benchmark Dataset for Multilingual Fact Checkingcited by 90
  16. 2022DataPerf: Benchmarks for Data-Centric AI Developmentcited by 89
  17. 2021CREAK: A Dataset for Commonsense Reasoning over Entity Knowledgecited by 88
  18. 2022Same Author or Just Same Topic? Towards Content-Independent Style Representationscited by 83
  19. 2022FLUTE: Figurative Language Understanding through Textual Explanationscited by 78
  20. 2021Contrastive Explanations for Model Interpretabilitycited by 75
  21. 2021DocNLI: A Large-scale Dataset for Document-level Natural Language Inferencecited by 75
  22. 2021Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Modelscited by 74
  23. 2023Efficient Benchmarking of Language Modelscited by 71
  24. 2022Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalizationcited by 69
  25. 2024Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question?cited by 68
  26. 2021Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristicscited by 67
  27. 2021"Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classificationcited by 66
  28. 2023Natural Language Reasoning, A Surveycited by 61
  29. 2022Generating Scientific Claims for Zero-Shot Scientific Fact Checkingcited by 59
  30. 2022Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictionscited by 57
  31. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 57
  32. 2021BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?cited by 56
  33. 2021A Systematic Investigation of Commonsense Knowledge in Large Language Modelscited by 56
  34. 2024Elements of World Knowledge (EWoK): A Cognition-Inspired Framework for Evaluating Basic World Knowledge in Language Modelscited by 55
  35. 2022Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasetscited by 54
  36. 2021Variational Information Bottleneck for Effective Low-Resource Fine-Tuningcited by 51
  37. 2021Fool Me Twice: Entailment from Wikipedia Gamificationcited by 50
  38. 2023Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrationscited by 49

50 most cited of 291 All 291 (JSON)