# NegBERT: A Transfer Learning Approach for Negation Detection and Scope Resolution

arXiv: https://arxiv.org/abs/1911.04211 · HTML: https://mlsanity.com/paper/1911.04211/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2019.

## Built on by (most cited first, 30 of 36)

- [Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese](https://mlsanity.com/paper/2211.01335/) (2022)
- [An Analysis of Negation in Natural Language Understanding Corpora](https://mlsanity.com/paper/2203.08929/) (2022)
- [CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation](https://mlsanity.com/paper/2211.00295/) (2022)
- [Know "No" Better: A Data-Driven Approach for Enhancing Negation Awareness in CLIP](https://mlsanity.com/paper/2501.10913/) (2025)
- [Not another Negation Benchmark: The NaN-NLI Test Suite for Sub-clausal Negation](https://mlsanity.com/paper/2210.03256/) (2022)
- [EventPlus: A Temporal Event Understanding Pipeline](https://mlsanity.com/paper/2101.04922/) (2021)
- [This is not correct! Negation-aware Evaluation of Language Generation Systems](https://mlsanity.com/paper/2307.13989/) (2023)
- [Learn to Understand Negation in Video Retrieval](https://mlsanity.com/paper/2205.00132/) (2022)
- [Improving negation detection with negation-focused pre-training](https://mlsanity.com/paper/2205.04012/) (2022)
- [Learning Disentangled Representations of Negation and Uncertainty](https://mlsanity.com/paper/2204.00511/) (2022)
- [Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models](https://mlsanity.com/paper/2305.17311/) (2023)
- [Paraphrasing in Affirmative Terms Improves Negation Understanding](https://mlsanity.com/paper/2406.07492/) (2024)
- [Revisiting Negation in Neural Machine Translation](https://mlsanity.com/paper/2107.12203/) (2021)
- [NADE: A Benchmark for Robust Adverse Drug Events Extraction in Face of Negations](https://mlsanity.com/paper/2109.10080/) (2021)
- [Thunder-NUBench: A Benchmark for LLMs' Sentence-Level Negation Understanding](https://mlsanity.com/paper/2506.14397/) (2025)
- [Machine Learning Approaches for Mental Illness Detection on Social Media: A Systematic Review of Biases and Methodological Challenges](https://mlsanity.com/paper/2410.16204/) (2024)
- [Increasing Adverse Drug Events extraction robustness on social media: case study on negation and speculation](https://mlsanity.com/paper/2209.02812/) (2022)
- [Resolving Legalese: A Multilingual Exploration of Negation Scope Resolution in Legal Documents](https://mlsanity.com/paper/2309.08695/) (2023)
- [Making Language Models Robust Against Negation](https://mlsanity.com/paper/2502.07717/) (2025)
- [Automatic Expansion of Domain-Specific Affective Models for Web Intelligence Applications](https://mlsanity.com/paper/2102.00827/) (2021)
- [Contradiction Detection in Persian Text](https://mlsanity.com/paper/2107.01987/) (2021)
- [ITTC @ TREC 2021 Clinical Trials Track](https://mlsanity.com/paper/2202.07858/) (2022)
- [Negation detection in Dutch clinical texts: an evaluation of rule-based and machine learning methods](https://mlsanity.com/paper/2209.00470/) (2022)
- [Assertion Detection Large Language Model In-context Learning LoRA Fine-tuning](https://mlsanity.com/paper/2401.17602/) (2024)
- [Logical Negation Augmenting and Debiasing for Prompt-based Methods](https://mlsanity.com/paper/2405.04872/) (2024)
- [Learning to Detect Cross-Modal Negation: An Analysis of Latent Representations and an Attention-Based Solution](https://mlsanity.com/paper/2607.17712/) (2026)
- [Negation Beyond the Verbal Channel: Temporal Multimodal Correlates in Dialogue](https://mlsanity.com/paper/2609.16396/) (2026)
- [Orthogonal Attention: A Cloze-Style Approach to Negation Scope Resolution](https://mlsanity.com/paper/2103.04294/) (2021)
- [Scope resolution of predicted negation cues: A two-step neural network-based approach](https://mlsanity.com/paper/2109.07264/) (2021)
- [Negation-Instance Based Evaluation of End-to-End Negation Resolution](https://mlsanity.com/paper/2109.10013/) (2021)

## Abstract (arXiv)

Negation is an important characteristic of language, and a major component of information extraction from text. This subtask is of considerable importance to the biomedical domain. Over the years, multiple approaches have been explored to address this problem: Rule-based systems, Machine Learning classifiers, Conditional Random Field Models, CNNs and more recently BiLSTMs. In this paper, we look at applying Transfer Learning to this problem. First, we extensively review previous literature addressing Negation Detection and Scope Resolution across the 3 datasets that have gained popularity over the years: the BioScope Corpus, the Sherlock dataset, and the SFU Review Corpus. We then explore the decision choices involved with using BERT, a popular transfer learning model, for this task, and report state-of-the-art results for scope resolution across all 3 datasets. Our model, referred to as NegBERT, achieves a token level F1 score on scope resolution of 92.36 on the Sherlock dataset, 95.68 on the BioScope Abstracts subcorpus, 91.24 on the BioScope Full Papers subcorpus, 90.95 on the SFU Review Corpus, outperforming the previous state-of-the-art systems by a significant margin. We also analyze the model's generalizability to datasets on which it is not trained.