# SciNLI: A Corpus for Natural Language Inference on Scientific Text

arXiv: https://arxiv.org/abs/2203.06728 · HTML: https://mlsanity.com/paper/2203.06728/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 42 references that are in the map.
- First posted on arXiv 13 Mar 2022.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [Annotation Artifacts in Natural Language Inference Data](https://mlsanity.com/paper/1803.02324/) (2018)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [Multi-Task Deep Neural Networks for Natural Language Understanding](https://mlsanity.com/paper/1901.11504/) (2019)
- [SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference](https://mlsanity.com/paper/1808.05326/) (2018)
- [Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?](https://mlsanity.com/paper/2005.00628/) (2020)
- [Guided Generation of Cause and Effect](https://mlsanity.com/paper/2107.09846/) (2021)
- [Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning](https://mlsanity.com/paper/1705.00557/) (2017)

## Built on by (most cited first, 27 of 27)

- [Matching Patients to Clinical Trials with Large Language Models](https://mlsanity.com/paper/2307.15051/) (2023)
- [Natural Language Reasoning, A Survey](https://mlsanity.com/paper/2303.14725/) (2023)
- [Scientific Fact-Checking: A Survey of Resources and Approaches](https://mlsanity.com/paper/2305.16859/) (2023)
- [Let's Learn Step by Step: Enhancing In-Context Learning Ability with Curriculum Learning](https://mlsanity.com/paper/2402.10738/) (2024)
- [PromptCBLUE: A Chinese Prompt Tuning Benchmark for the Medical Domain](https://mlsanity.com/paper/2310.14151/) (2023)
- [SciER: An Entity and Relation Extraction Dataset for Datasets, Methods, and Tasks in Scientific Documents](https://mlsanity.com/paper/2410.21155/) (2024)
- [The ACL OCL Corpus: Advancing Open Science in Computational Linguistics](https://mlsanity.com/paper/2305.14996/) (2023)
- [Epistemology of Language Models: Do Language Models Have Holistic Knowledge?](https://mlsanity.com/paper/2403.12862/) (2024)
- [MSciNLI: A Diverse Benchmark for Scientific Natural Language Inference](https://mlsanity.com/paper/2404.08066/) (2024)
- [A Novel Cartography-Based Curriculum Learning Method Applied on RoNLI: The First Romanian Natural Language Inference Corpus](https://mlsanity.com/paper/2405.11877/) (2024)
- [Agree, Disagree, Explain: Decomposing Human Label Variation in NLI through the Lens of Explanations](https://mlsanity.com/paper/2510.16458/) (2025)
- [SciPrompt: Knowledge-augmented Prompting for Fine-grained Categorization of Scientific Topics](https://mlsanity.com/paper/2410.01946/) (2024)
- [How Hard is this Test Set? NLI Characterization by Exploiting Training Dynamics](https://mlsanity.com/paper/2410.03429/) (2024)
- [Unsupervised Knowledge Graph Construction and Event-centric Knowledge Infusion for Scientific NLI](https://mlsanity.com/paper/2210.15248/) (2022)
- [Co-training for Low Resource Scientific Natural Language Inference](https://mlsanity.com/paper/2406.14666/) (2024)
- [FinNLI: Novel Dataset for Multi-Genre Financial Natural Language Inference Benchmarking](https://mlsanity.com/paper/2504.16188/) (2025)
- [Validity Assessment of Legal Will Statements as Natural Language Inference](https://mlsanity.com/paper/2210.16989/) (2022)
- [Talking with Machines: A Comprehensive Survey of Emergent Dialogue Systems](https://mlsanity.com/paper/2305.16324/) (2023)
- [MapExplorer: New Content Generation from Low-Dimensional Visualizations](https://mlsanity.com/paper/2412.18673/) (2024)
- [ScholarBench: A Bilingual Benchmark for Abstraction, Comprehension, and Reasoning Evaluation in Academic Contexts](https://mlsanity.com/paper/2505.16566/) (2025)
- [Explaining Patterns in Data with Language Models via Interpretable Autoprompting](https://mlsanity.com/paper/2210.01848/) (2022)
- [SciMRC: Multi-perspective Scientific Machine Reading Comprehension](https://mlsanity.com/paper/2306.14149/) (2023)
- [ESNLIR: A Spanish Multi-Genre Dataset with Causal Relationships](https://mlsanity.com/paper/2503.08803/) (2025)
- [A MISMATCHED Benchmark for Scientific Natural Language Inference](https://mlsanity.com/paper/2506.04603/) (2025)
- [A Parameter-Efficient Transfer Learning Approach through Multitask Prompt Distillation and Decomposition for Clinical NLP](https://mlsanity.com/paper/2604.06650/) (2026)
- [Structurally-bounded Agentic Graph Exploration for Evidence-Grounded Scholarly DeepSearch](https://mlsanity.com/paper/2608.24809/) (2026)
- [RATIO: A Benchmark for Retrieval Across Typed Ideation Operations in Scientific Literature](https://mlsanity.com/paper/2608.27394/) (2026)

## Abstract (arXiv)

Existing Natural Language Inference (NLI) datasets, while being instrumental in the advancement of Natural Language Understanding (NLU) research, are not related to scientific text. In this paper, we introduce SciNLI, a large dataset for NLI that captures the formality in scientific text and contains 107,412 sentence pairs extracted from scholarly papers on NLP and computational linguistics. Given that the text used in scientific literature differs vastly from the text used in everyday language both in terms of vocabulary and sentence structure, our dataset is well suited to serve as a benchmark for the evaluation of scientific NLU models. Our experiments show that SciNLI is harder to classify than the existing NLI datasets. Our best performing model with XLNet achieves a Macro F1 score of only 78.18% and an accuracy of 78.23% showing that there is substantial room for improvement.