Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches
Not on the map. Why?
Abstract (arXiv)
In the NLP community, recent years have seen a surge of research activities that address machines' ability to perform deep language understanding which goes beyond what is explicitly stated in text, rather relying on reasoning and knowledge of the world. Many benchmark tasks and datasets have been created to support the development and evaluation of such natural language inference ability. As these benchmarks become instrumental and a driving force for the NLP research community, this paper aims to provide an overview of recent benchmarks, relevant knowledge resources, and state-of-the-art learning and inference approaches in order to support a better understanding of this growing field.
Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Apr 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 10; 2022: 11; 2023: 18; 2024: 13; 2025: 10; 2026: 2.
64 papers.
Sort:
By area:
- 2022Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusterscited by 44
- 2024Training on the Test Task Confounds Evaluation and Emergencecited by 33
- 2025Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challengescited by 31
- 2024Inherent Trade-Offs between Diversity and Stability in Multi-Task Benchmarkscited by 26
- 2022The Defeat of the Winograd Schema Challengecited by 25
- 2023It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performancecited by 25
- 2021Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledgecited by 24
- 2023Inductive reasoning in humans and large language modelscited by 23
- 2021Automated scholarly paper review: Concepts, technologies, and challengescited by 21
- 2023Is BERT Blind? Exploring the Effect of Vision-and-Language Pretraining on Visual Language Understandingcited by 19
- 2022Assemble Foundation Models for Automatic Code Summarizationcited by 17
- 2023Gemini in Reasoning: Unveiling Commonsense in Multimodal Large Language Modelscited by 15
38 more
- 2024Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutionscited by 15
- 2021Benchmarking for Biomedical Natural Language Processing Tasks with a Domain Specific ALBERTcited by 14
- 2022A new hope for network model generalizationcited by 11
- 2023Abductive Commonsense Reasoning Exploiting Mutually Exclusive Explanationscited by 11
- 2024Resolving Word Vagueness with Scenario-guided Adapter for Natural Language Inferencecited by 11
- 2022Towards Large-Scale Interpretable Knowledge Graph Reasoning for Dialogue Systemscited by 10
- 2024Exploring the Limitations of Large Language Models in Compositional Relation Reasoningcited by 9
- 2022Logiformer: A Two-Branch Graph Transformer Network for Interpretable Logical Reasoningcited by 8
- 2023CRoW: Benchmarking Commonsense Reasoning in Real-World Taskscited by 8
- 2021SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoningcited by 7
- 2023The Web Can Be Your Oyster for Improving Large Language Modelscited by 7
- 2023Conditions for Length Generalization in Learning Reasoning Skillscited by 7
- 2024From Data to Commonsense Reasoning: The Use of Large Language Models for Explainable AIcited by 7
- 2022KMIR: A Benchmark for Evaluating Knowledge Memorization, Identification and Reasoning Abilities of Language Modelscited by 6
- 2025Continual Pre-training of MoEs: How robust is your router?cited by 5
- 2021Context-Aware Interaction Network for Question Matchingcited by 4
- 2021Natural Language Processing with Commonsense Knowledge: A Surveycited by 4
- 2023SDOH-NLI: a Dataset for Inferring Social Determinants of Health from Clinical Notescited by 4
- 2024How Hard is this Test Set? NLI Characterization by Exploiting Training Dynamicscited by 4
- 2024Transparent and Coherent Procedural Mistake Detectioncited by 4
- 2025LOGicalThought: Logic-Based Ontological Grounding of LLMs for High-Assurance Reasoningcited by 4
- 2021Classifying vaccine sentiment tweets by modelling domain-specific representation and commonsense knowledge into context-aware attentive GRUcited by 3
- 2023RecycleGPT: An Autoregressive Language Model with Recyclable Modulecited by 3
- 2023CORECODE: A Common Sense Annotated Dialogue Dataset with Benchmark Tasks for Chinese Large Language Modelscited by 3
- 2024The Base-Rate Effect on LLM Benchmark Performance: Disambiguating Test-Taking Strategies from Benchmark Performancecited by 3
- 2025EtiCor++: Towards Understanding Etiquettical Bias in LLMscited by 3
- 2021Projection: A Mechanism for Human-like Reasoning in Artificial Intelligencecited by 2
- 2022On Reality and the Limits of Language Data: Aligning LLMs with Human Normscited by 2
- 2023Mitigating Data Scarcity for Large Language Modelscited by 2
- 2023Complementary Advantages of ChatGPTs and Human Readers in Reasoning: Evidence from English Text Reading Comprehensioncited by 2
- 2024Temporal Validity Change Predictioncited by 2
- 2025MorphNLI: A Stepwise Approach to Natural Language Inference Using Text Morphingcited by 2
- 2025Not quite Sherlock Holmes: Language model predictions do not reliably differentiate impossible from improbable eventscited by 2
- 2022Tri-Attention: Explicit Context-Aware Attention Mechanism for Natural Language Processingcited by 1
- 2022ZeroKBC: A Comprehensive Benchmark for Zero-Shot Knowledge Base Completioncited by 1
- 2025Capturing Symmetry and Antisymmetry in Language Models through Symmetry-Aware Training Objectivescited by 1
- 2026Physical Commonsense Reasoning for Lower-Resourced Languages and Dialects: a Study on Basquecited by 1
- 2026Indirect Question Answering in English, German and Bavarian: A Challenging Task for High- and Low-Resource Languages Alikecited by 1
50 most cited of 64 All 64 (JSON)
- 2026Indirect Question Answering in English, German and Bavarian: A Challenging Task for High- and Low-Resource Languages Alike16 Mar 2026
- 2026Physical Commonsense Reasoning for Lower-Resourced Languages and Dialects: a Study on Basque16 Feb 2026
- 2025LOGicalThought: Logic-Based Ontological Grounding of LLMs for High-Assurance Reasoning2 Oct 2025
- 2025What Are Research Hypotheses?29 Aug 2025
- 2025Commonsense on Demand: Generating and Selectively Integrating Commonsense Knowledge for Natural Language Inference20 Jul 2025
- 2025EtiCor++: Towards Understanding Etiquettical Bias in LLMs10 Jun 2025
- 2025Not quite Sherlock Holmes: Language model predictions do not reliably differentiate impossible from improbable events7 Jun 2025
- 2025Capturing Symmetry and Antisymmetry in Language Models through Symmetry-Aware Training Objectives22 Apr 2025
- 2025Continual Pre-training of MoEs: How robust is your router?6 Mar 2025
- 2025Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challenges17 Feb 2025
- 2025MorphNLI: A Stepwise Approach to Natural Language Inference Using Text Morphing13 Feb 2025
- 2025Neuro-Symbolic Contrastive Learning for Cross-domain Inference13 Feb 2025
38 more
- 2024Transparent and Coherent Procedural Mistake Detection16 Dec 2024
- 2024Health Misinformation in Social Networks: A Survey of IT Approaches24 Oct 2024
- 2024How Hard is this Test Set? NLI Characterization by Exploiting Training Dynamics4 Oct 2024
- 2024Training on the Test Task Confounds Evaluation and Emergence10 Jul 2024
- 2024From Data to Commonsense Reasoning: The Use of Large Language Models for Explainable AI4 Jul 2024
- 2024The Base-Rate Effect on LLM Benchmark Performance: Disambiguating Test-Taking Strategies from Benchmark Performance17 Jun 2024
- 2024Improving Commonsense Bias Classification by Mitigating the Influence of Demographic Terms11 Jun 2024
- 2024Resolving Word Vagueness with Scenario-guided Adapter for Natural Language Inference21 May 2024
- 2024Inherent Trade-Offs between Diversity and Stability in Multi-Task Benchmarks2 May 2024
- 2024Projective Methods for Mitigating Gender Bias in Pre-trained Language Models27 Mar 2024
- 2024Exploring the Limitations of Large Language Models in Compositional Relation Reasoning5 Mar 2024
- 2024Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutions2 Feb 2024
- 2024Temporal Validity Change Prediction1 Jan 2024
- 2023Gemini in Reasoning: Unveiling Commonsense in Multimodal Large Language Models29 Dec 2023
- 2023CORECODE: A Common Sense Annotated Dialogue Dataset with Benchmark Tasks for Chinese Large Language Models20 Dec 2023
- 2023A Generic NLI approach for Classification of Sentiment Associated with Therapies28 Nov 2023
- 2023Conditions for Length Generalization in Learning Reasoning Skills22 Nov 2023
- 2023Complementary Advantages of ChatGPTs and Human Readers in Reasoning: Evidence from English Text Reading Comprehension17 Nov 2023
- 2023Perturbation-based Active Learning for Question Answering4 Nov 2023
- 2023SDOH-NLI: a Dataset for Inferring Social Determinants of Health from Clinical Notes27 Oct 2023
- 2023CRoW: Benchmarking Commonsense Reasoning in Real-World Tasks23 Oct 2023
- 2023Link Prediction for Wikipedia Articles as a Natural Language Inference Task31 Aug 2023
- 2023RecycleGPT: An Autoregressive Language Model with Recyclable Module7 Aug 2023
- 2023Inductive reasoning in humans and large language models11 Jun 2023
- 2023Abductive Commonsense Reasoning Exploiting Mutually Exclusive Explanations24 May 2023
- 2023The Web Can Be Your Oyster for Improving Large Language Models18 May 2023
- 2023It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance15 May 2023
- 2023Profiling the news spreading barriers using news headlines7 Apr 2023
- 2023Is BERT Blind? Exploring the Effect of Vision-and-Language Pretraining on Visual Language Understanding21 Mar 2023
- 2023CLICKER: Attention-Based Cross-Lingual Commonsense Knowledge Transfer26 Feb 2023
- 2023Mitigating Data Scarcity for Large Language Models3 Feb 2023
- 2022ZeroKBC: A Comprehensive Benchmark for Zero-Shot Knowledge Base Completion6 Dec 2022
- 2022A Probabilistic-Logic based Commonsense Representation Framework for Modelling Inferences with Multiple Antecedents and Varying Likelihoods30 Nov 2022
- 2022Tri-Attention: Explicit Context-Aware Attention Mechanism for Natural Language Processing5 Nov 2022
- 2022On Reality and the Limits of Language Data: Aligning LLMs with Human Norms25 Aug 2022
- 2022A new hope for network model generalization12 Jul 2022
- 2022Logiformer: A Two-Branch Graph Transformer Network for Interpretable Logical Reasoning2 May 2022
- 2022Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters15 Apr 2022
50 newest of 64 All 64 (JSON)
- 2023Inductive reasoning in humans and large language modelscited by 23
- 2023Is BERT Blind? Exploring the Effect of Vision-and-Language Pretraining on Visual Language Understandingcited by 19
- 2023Gemini in Reasoning: Unveiling Commonsense in Multimodal Large Language Modelscited by 15
- 2024Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutionscited by 15
- 2023Abductive Commonsense Reasoning Exploiting Mutually Exclusive Explanationscited by 11
- 2022Towards Large-Scale Interpretable Knowledge Graph Reasoning for Dialogue Systemscited by 10
- 2024Exploring the Limitations of Large Language Models in Compositional Relation Reasoningcited by 9
- 2022Logiformer: A Two-Branch Graph Transformer Network for Interpretable Logical Reasoningcited by 8
- 2023CRoW: Benchmarking Commonsense Reasoning in Real-World Taskscited by 8
- 2023Conditions for Length Generalization in Learning Reasoning Skillscited by 7
- 2024From Data to Commonsense Reasoning: The Use of Large Language Models for Explainable AIcited by 7
- 2022KMIR: A Benchmark for Evaluating Knowledge Memorization, Identification and Reasoning Abilities of Language Modelscited by 6
12 most cited of 17 All 17 (JSON)
- 2026Physical Commonsense Reasoning for Lower-Resourced Languages and Dialects: a Study on Basque16 Feb 2026
- 2025LOGicalThought: Logic-Based Ontological Grounding of LLMs for High-Assurance Reasoning2 Oct 2025
- 2024From Data to Commonsense Reasoning: The Use of Large Language Models for Explainable AI4 Jul 2024
- 2024Exploring the Limitations of Large Language Models in Compositional Relation Reasoning5 Mar 2024
- 2024Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutions2 Feb 2024
- 2023Gemini in Reasoning: Unveiling Commonsense in Multimodal Large Language Models29 Dec 2023
- 2023CORECODE: A Common Sense Annotated Dialogue Dataset with Benchmark Tasks for Chinese Large Language Models20 Dec 2023
- 2023Conditions for Length Generalization in Learning Reasoning Skills22 Nov 2023
- 2023Complementary Advantages of ChatGPTs and Human Readers in Reasoning: Evidence from English Text Reading Comprehension17 Nov 2023
- 2023CRoW: Benchmarking Commonsense Reasoning in Real-World Tasks23 Oct 2023
- 2023Inductive reasoning in humans and large language models11 Jun 2023
- 2023Abductive Commonsense Reasoning Exploiting Mutually Exclusive Explanations24 May 2023
12 newest of 17 All 17 (JSON)
- 2022Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusterscited by 44
- 2024Training on the Test Task Confounds Evaluation and Emergencecited by 33
- 2025Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challengescited by 31
- 2024Inherent Trade-Offs between Diversity and Stability in Multi-Task Benchmarkscited by 26
- 2022The Defeat of the Winograd Schema Challengecited by 25
- 2023It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performancecited by 25
- 2021Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledgecited by 24
- 2021Automated scholarly paper review: Concepts, technologies, and challengescited by 21
- 2022Assemble Foundation Models for Automatic Code Summarizationcited by 17
- 2021Benchmarking for Biomedical Natural Language Processing Tasks with a Domain Specific ALBERTcited by 14
- 2022A new hope for network model generalizationcited by 11
- 2024Resolving Word Vagueness with Scenario-guided Adapter for Natural Language Inferencecited by 11
12 most cited of 46 All 46 (JSON)
- 2026Indirect Question Answering in English, German and Bavarian: A Challenging Task for High- and Low-Resource Languages Alike16 Mar 2026
- 2025What Are Research Hypotheses?29 Aug 2025
- 2025Commonsense on Demand: Generating and Selectively Integrating Commonsense Knowledge for Natural Language Inference20 Jul 2025
- 2025EtiCor++: Towards Understanding Etiquettical Bias in LLMs10 Jun 2025
- 2025Not quite Sherlock Holmes: Language model predictions do not reliably differentiate impossible from improbable events7 Jun 2025
- 2025Capturing Symmetry and Antisymmetry in Language Models through Symmetry-Aware Training Objectives22 Apr 2025
- 2025Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challenges17 Feb 2025
- 2025MorphNLI: A Stepwise Approach to Natural Language Inference Using Text Morphing13 Feb 2025
- 2025Neuro-Symbolic Contrastive Learning for Cross-domain Inference13 Feb 2025
- 2024Transparent and Coherent Procedural Mistake Detection16 Dec 2024
- 2024Health Misinformation in Social Networks: A Survey of IT Approaches24 Oct 2024
- 2024How Hard is this Test Set? NLI Characterization by Exploiting Training Dynamics4 Oct 2024
12 newest of 46 All 46 (JSON)