mlsanityData to 7 Oct 2026

Improving negation detection with negation-focused pre-training

arXiv 2205.04012 ↗First posted 9 May 2022cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

Negation is a common linguistic feature that is crucial in many language understanding tasks, yet it remains a hard problem due to diversity in its expression in different types of text. Recent work has shown that state-of-the-art NLP models underperform on samples containing negation in various tasks, and that negation detection models do not transfer well across domains. We propose a new negation-focused pre-training strategy, involving targeted data augmentation and negation masking, to better incorporate negation information into language models. Extensive experiments on common benchmarks show that our proposed approach improves negation detection performance and generalizability over the strong baseline NegBERT (Khandewal and Sawant, 2020).

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 10 of its 28 references that are in the map. First posted on arXiv 9 May 2022.

Builds on

10 of 28 references.

References · 10

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2016Pointer Sentinel Mixture Modelscited by 2,992
  4. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  5. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  6. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  7. 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
  8. 2017NegBio: a high-performance tool for negation and uncertainty detection in radiology reportscited by 79
  9. 2019NegBERT: A Transfer Learning Approach for Negation Detection and Scope Resolutioncited by 36
  10. 2019Improving Sentiment Analysis with Multi-task Learning of Negationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 5; 2024: 3; 2025: 2; 2026: 2.

14 papers.

Sort:

By area:

  1. 2023Language models are not naysayers: An analysis of language models on negation benchmarkscited by 73
  2. 2023Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLUcited by 53
  3. 2022CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negationcited by 38
  4. 2022Not another Negation Benchmark: The NaN-NLI Test Suite for Sub-clausal Negationcited by 24
  5. 2023This is not correct! Negation-aware Evaluation of Language Generation Systemscited by 16
  6. 2023Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Modelscited by 12
  7. 2023Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blindscited by 7
  8. 2024Paraphrasing in Affirmative Terms Improves Negation Understandingcited by 7
  9. 2024Are LLMs Models of Distributional Semantics? A Case Study on Quantifierscited by 3
  10. 2025Making Language Models Robust Against Negationcited by 3
  11. 2025Semantic Adapter for Universal Text Embeddings: Diagnosing and Mitigating Negation Blindness to Enhance Universalitycited by 1
  12. 2026Learning to Detect Cross-Modal Negation: An Analysis of Latent Representations and an Attention-Based Solutioncited by 1
2 more
  1. 2024Negation Triplet Extraction with Syntactic Dependency and Semantic Consistencycited by 0
  2. 2026Not all Negation Cues are Equal: Affixal Negations Yield Better Negation Understandingcited by 0