# A negation detection assessment of GPTs: analysis with the xNot360 dataset

arXiv: https://arxiv.org/abs/2306.16638 · HTML: https://mlsanity.com/paper/2306.16638/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 29 Jun 2023.

## Builds on (most cited first, 13 of 13)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)
- [Abductive Commonsense Reasoning](https://mlsanity.com/paper/1908.05739/) (2019)
- [Explainable Prediction of Medical Codes from Clinical Text](https://mlsanity.com/paper/1802.05695/) (2018)
- [Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4](https://mlsanity.com/paper/2304.03439/) (2023)
- [Understanding by Understanding Not: Modeling Negation in Language Models](https://mlsanity.com/paper/2105.03519/) (2021)
- [An Analysis of Negation in Natural Language Understanding Corpora](https://mlsanity.com/paper/2203.08929/) (2022)
- [Visual Abductive Reasoning](https://mlsanity.com/paper/2203.14040/) (2022)
- [A multi-component framework for the analysis and design of explainable artificial intelligence](https://mlsanity.com/paper/2005.01908/) (2020)
- [Attentive Deep Neural Networks for Legal Document Retrieval](https://mlsanity.com/paper/2212.13899/) (2022)

## Built on by (most cited first, 10 of 10)

- [Black-Box Analysis: GPTs Across Time in Legal Textual Entailment Task](https://mlsanity.com/paper/2309.05501/) (2023)
- [Enhancing Logical Reasoning in Large Language Models to Facilitate Legal Applications](https://mlsanity.com/paper/2311.13095/) (2023)
- [Thunder-NUBench: A Benchmark for LLMs' Sentence-Level Negation Understanding](https://mlsanity.com/paper/2506.14397/) (2025)
- [Do large language models and humans have similar behaviors in causal inference with script knowledge?](https://mlsanity.com/paper/2311.07311/) (2023)
- [Balancing Exploration and Exploitation in LLM using Soft RLLF for Enhanced Negation Understanding](https://mlsanity.com/paper/2403.01185/) (2024)
- [KRAG Framework for Enhancing LLMs in the Legal Domain](https://mlsanity.com/paper/2410.07551/) (2024)
- [BIS Reasoning 1.0: The First Large-Scale Japanese Benchmark for Belief-Inconsistent Syllogistic Reasoning](https://mlsanity.com/paper/2506.06955/) (2025)
- [VLSP 2023 -- LTER: A Summary of the Challenge on Legal Textual Entailment Recognition](https://mlsanity.com/paper/2403.03435/) (2024)
- [Interpreting Negation in GPT-2: Layer- and Head-Level Causal Analysis](https://mlsanity.com/paper/2603.12423/) (2026)
- [Not What You Meant: Can LLMs Follow a Specified Negation Semantics?](https://mlsanity.com/paper/2609.27517/) (2026)

## Abstract (arXiv)

Negation is a fundamental aspect of natural language, playing a critical role in communication and comprehension. Our study assesses the negation detection performance of Generative Pre-trained Transformer (GPT) models, specifically GPT-2, GPT-3, GPT-3.5, and GPT-4. We focus on the identification of negation in natural language using a zero-shot prediction approach applied to our custom xNot360 dataset. Our approach examines sentence pairs labeled to indicate whether the second sentence negates the first. Our findings expose a considerable performance disparity among the GPT models, with GPT-4 surpassing its counterparts and GPT-3.5 displaying a marked performance reduction. The overall proficiency of the GPT models in negation detection remains relatively modest, indicating that this task pushes the boundaries of their natural language understanding capabilities. We not only highlight the constraints of GPT models in handling negation but also emphasize the importance of logical reliability in high-stakes domains such as healthcare, science, and law.