# Lessons from the Use of Natural Language Inference (NLI) in Requirements Engineering Tasks

arXiv: https://arxiv.org/abs/2405.05135 · HTML: https://mlsanity.com/paper/2405.05135/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 63 references that are in the map.
- First posted on arXiv 24 Apr 2024.

## Builds on (most cited first, 21 of 21)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://mlsanity.com/paper/1909.00161/) (2019)
- [GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Records](https://mlsanity.com/paper/2203.03540/) (2022)
- [Enhanced LSTM for Natural Language Inference](https://mlsanity.com/paper/1609.06038/) (2016)
- [The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink](https://mlsanity.com/paper/2204.05149/) (2022)
- [Lessons from Natural Language Inference in the Clinical Domain](https://mlsanity.com/paper/1808.06752/) (2018)
- [Entailment as Few-Shot Learner](https://mlsanity.com/paper/2104.14690/) (2021)
- [Transforming Question Answering Datasets Into Natural Language Inference Datasets](https://mlsanity.com/paper/1809.02922/) (2018)
- [Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition](https://mlsanity.com/paper/2004.03066/) (2020)
- [Label Verbalization and Entailment for Effective Zero- and Few-Shot Relation Extraction](https://mlsanity.com/paper/2109.03659/) (2021)
- [Semantic Diversity in Dialogue with Natural Language Inference](https://mlsanity.com/paper/2205.01497/) (2022)
- [Zero-Shot Text Classification with Self-Training](https://mlsanity.com/paper/2210.17541/) (2022)
- [Natural Language Inference Prompts for Zero-shot Emotion Classification in Text across Corpora](https://mlsanity.com/paper/2209.06701/) (2022)
- [Fake News Detection as Natural Language Inference](https://mlsanity.com/paper/1907.07347/) (2019)
- [What Works Better? A Study of Classifying Requirements](https://mlsanity.com/paper/1707.02358/) (2017)
- [Empirical Evaluation of ChatGPT on Requirements Information Retrieval Under Zero-Shot Setting](https://mlsanity.com/paper/2304.12562/) (2023)
- [Natural Language Inference with Self-Attention for Veracity Assessment of Pandemic Claims](https://mlsanity.com/paper/2205.02596/) (2022)
- [Rethinking STS and NLI in Large Language Models](https://mlsanity.com/paper/2309.08969/) (2023)
- [Evaluating the Effectiveness of Natural Language Inference for Hate Speech Detection in Languages with Limited Labeled Data](https://mlsanity.com/paper/2306.03722/) (2023)

## Built on by (most cited first, 12 of 12)

- [Explainable Compliance Detection with Multi-Hop Natural Language Inference on Assurance Case Structure](https://mlsanity.com/paper/2506.08713/) (2025)
- [Semantic Commit: Helping Users Update Intent Specifications for AI Memory at Scale](https://mlsanity.com/paper/2504.09283/) (2025)
- [ARLO: A Tailorable Approach for Transforming Natural Language Software Requirements into Architecture using LLMs](https://mlsanity.com/paper/2504.06143/) (2025)
- [Synthline: A Product Line Approach for Synthetic Requirements Engineering Data Generation using Large Language Models](https://mlsanity.com/paper/2505.03265/) (2025)
- [Faithful Autoformalization of Natural Language Assertions](https://mlsanity.com/paper/2607.13303/) (2026)
- [Systems Engineering for Autonomous Vehicles; Supervising AI using Large Language Models (SSuperLLM)](https://mlsanity.com/paper/2501.10839/) (2025)
- [QualiTagger: Automating software quality detection in issue trackers](https://mlsanity.com/paper/2504.11053/) (2025)
- [Multi-Sample Prompting and Actor-Critic Prompt Optimization for Diverse Synthetic Data Generation](https://mlsanity.com/paper/2506.21138/) (2025)
- [Towards an Agentic LLM-based Approach to Requirement Formalization from Unstructured Specifications](https://mlsanity.com/paper/2604.18228/) (2026)
- [Cross-Domain Data Selection and Augmentation for Automatic Compliance Detection](https://mlsanity.com/paper/2604.21469/) (2026)
- [ArgRE: Formal Argumentation for Conflict Resolution in Multi-Agent Requirements Negotiation](https://mlsanity.com/paper/2604.23124/) (2026)
- [ReqGenX: An Empirical Study of Atomic Decomposition, Artifact Regeneration, and Reconstruction for Legacy SRS Documents](https://mlsanity.com/paper/2607.16564/) (2026)

## Abstract (arXiv)

We investigate the use of Natural Language Inference (NLI) in automating requirements engineering tasks. In particular, we focus on three tasks: requirements classification, identification of requirements specification defects, and detection of conflicts in stakeholders' requirements. While previous research has demonstrated significant benefit in using NLI as a universal method for a broad spectrum of natural language processing tasks, these advantages have not been investigated within the context of software requirements engineering. Therefore, we design experiments to evaluate the use of NLI in requirements analysis. We compare the performance of NLI with a spectrum of approaches, including prompt-based models, conventional transfer learning, Large Language Models (LLMs)-powered chatbot models, and probabilistic models. Through experiments conducted under various learning settings including conventional learning and zero-shot, we demonstrate conclusively that our NLI method surpasses classical NLP methods as well as other LLMs-based and chatbot models in the analysis of requirements specifications. Additionally, we share lessons learned characterizing the learning settings that make NLI a suitable approach for automating requirements engineering tasks.