mlsanityData to 7 Oct 2026

SciNLI: A Corpus for Natural Language Inference on Scientific Text

arXiv 2203.06728 ↗First posted 13 Mar 2022cs.CLcited by 27

Not on the map. Why?

Abstract (arXiv)

Existing Natural Language Inference (NLI) datasets, while being instrumental in the advancement of Natural Language Understanding (NLU) research, are not related to scientific text. In this paper, we introduce SciNLI, a large dataset for NLI that captures the formality in scientific text and contains 107,412 sentence pairs extracted from scholarly papers on NLP and computational linguistics. Given that the text used in scientific literature differs vastly from the text used in everyday language both in terms of vocabulary and sentence structure, our dataset is well suited to serve as a benchmark for the evaluation of scientific NLU models. Our experiments show that SciNLI is harder to classify than the existing NLI datasets. Our best performing model with XLNet achieves a Macro F1 score of only 78.18% and an accuracy of 78.23% showing that there is substantial room for improvement.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 18 of its 42 references that are in the map. First posted on arXiv 13 Mar 2022.

Builds on

18 of 42 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  5. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  6. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  7. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  8. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  9. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  10. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  11. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  12. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
6 more
  1. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 428
  2. 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 349
  3. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  4. 2020Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?cited by 127
  5. 2021Guided Generation of Cause and Effectcited by 33
  6. 2017Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learningcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 7; 2024: 9; 2025: 5; 2026: 3.

27 papers.

Sort:

By area:

  1. 2023Matching Patients to Clinical Trials with Large Language Modelscited by 68
  2. 2023Natural Language Reasoning, A Surveycited by 61
  3. 2023Scientific Fact-Checking: A Survey of Resources and Approachescited by 53
  4. 2024Let's Learn Step by Step: Enhancing In-Context Learning Ability with Curriculum Learningcited by 47
  5. 2023PromptCBLUE: A Chinese Prompt Tuning Benchmark for the Medical Domaincited by 40
  6. 2024SciER: An Entity and Relation Extraction Dataset for Datasets, Methods, and Tasks in Scientific Documentscited by 20
  7. 2023The ACL OCL Corpus: Advancing Open Science in Computational Linguisticscited by 16
  8. 2024Epistemology of Language Models: Do Language Models Have Holistic Knowledge?cited by 9
  9. 2024MSciNLI: A Diverse Benchmark for Scientific Natural Language Inferencecited by 8
  10. 2024A Novel Cartography-Based Curriculum Learning Method Applied on RoNLI: The First Romanian Natural Language Inference Corpuscited by 8
  11. 2025Agree, Disagree, Explain: Decomposing Human Label Variation in NLI through the Lens of Explanationscited by 5
  12. 2024SciPrompt: Knowledge-augmented Prompting for Fine-grained Categorization of Scientific Topicscited by 4
15 more
  1. 2024How Hard is this Test Set? NLI Characterization by Exploiting Training Dynamicscited by 4
  2. 2022Unsupervised Knowledge Graph Construction and Event-centric Knowledge Infusion for Scientific NLIcited by 3
  3. 2024Co-training for Low Resource Scientific Natural Language Inferencecited by 3
  4. 2025FinNLI: Novel Dataset for Multi-Genre Financial Natural Language Inference Benchmarkingcited by 3
  5. 2022Validity Assessment of Legal Will Statements as Natural Language Inferencecited by 1
  6. 2024MapExplorer: New Content Generation from Low-Dimensional Visualizationscited by 1
  7. 2025ScholarBench: A Bilingual Benchmark for Abstraction, Comprehension, and Reasoning Evaluation in Academic Contextscited by 1
  8. 2022Explaining Patterns in Data with Language Models via Interpretable Autopromptingcited by 0
  9. 2023Talking with Machines: A Comprehensive Survey of Emergent Dialogue Systemscited by 0
  10. 2023SciMRC: Multi-perspective Scientific Machine Reading Comprehensioncited by 0
  11. 2025ESNLIR: A Spanish Multi-Genre Dataset with Causal Relationshipscited by 0
  12. 2025A MISMATCHED Benchmark for Scientific Natural Language Inferencecited by 0
  13. 2026A Parameter-Efficient Transfer Learning Approach through Multitask Prompt Distillation and Decomposition for Clinical NLPcited by 0
  14. 2026Structurally-bounded Agentic Graph Exploration for Evidence-Grounded Scholarly DeepSearchcited by 0
  15. 2026RATIO: A Benchmark for Retrieval Across Typed Ideation Operations in Scientific Literaturecited by 0