mlsanityData to 7 Oct 2026

An Analysis of Negation in Natural Language Understanding Corpora

arXiv 2203.08929 ↗First posted 16 Mar 2022cs.CLcited by 39

Not on the map. Why?

Abstract (arXiv)

This paper analyzes negation in eight popular corpora spanning six natural language understanding tasks. We show that these corpora have few negations compared to general-purpose English, and that the few negations in them are often unimportant. Indeed, one can often ignore negations and still make the right predictions. Additionally, experimental results show that state-of-the-art transformers trained with these corpora obtain substantially worse results with instances that contain negation, especially if the negations are important. We conclude that new corpora accounting for negation are needed to solve natural language understanding tasks when negation is present.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 19 of its 37 references that are in the map. First posted on arXiv 16 Mar 2022.

Builds on

19 of 37 references.

References · 19

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  5. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  6. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  7. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  8. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  9. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  10. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  11. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  12. 2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 370
7 more
  1. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  2. 2018Stress Test Evaluation for Natural Language Inferencecited by 211
  3. 2019Coreference Resolution as Query-based Span Predictioncited by 91
  4. 2019Investigating BERT's Knowledge of Language: Five Analysis Methods with NPIscited by 58
  5. 2020jiant: A Software Toolkit for Research on General-Purpose Text Understanding Modelscited by 39
  6. 2019NegBERT: A Transfer Learning Approach for Negation Detection and Scope Resolutioncited by 36
  7. 2019HELP: A Dataset for Identifying Shortcomings of Neural Models in Monotonicity Reasoningcited by 35

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 16; 2024: 8; 2025: 7; 2026: 4.

39 papers.

Sort:

By area:

  1. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 290
  2. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  3. 2023Language models are not naysayers: An analysis of language models on negation benchmarkscited by 73
  4. 2022CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negationcited by 38
  5. 2023Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledgecited by 32
  6. 2023NevIR: Negation in Neural Information Retrievalcited by 28
  7. 2023It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performancecited by 25
  8. 2022Not another Negation Benchmark: The NaN-NLI Test Suite for Sub-clausal Negationcited by 24
  9. 2022On the Limitations of Dataset Balancing: The Lost Battle Against Spurious Correlationscited by 19
  10. 2023This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Modelscited by 18
  11. 2024Investigating and Addressing Hallucinations of LLMs in Tasks Involving Negationcited by 15
  12. 2023Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Modelscited by 12
27 more
  1. 2023A negation detection assessment of GPTs: analysis with the xNot360 datasetcited by 10
  2. 2022Leveraging Affirmative Interpretations from Negation Improves Natural Language Understandingcited by 7
  3. 2023Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blindscited by 7
  4. 2024Paraphrasing in Affirmative Terms Improves Negation Understandingcited by 7
  5. 2024The Dark Side of Rich Rewards: Understanding and Mitigating Noise in VLM Rewardscited by 7
  6. 2025Multi-Agent LLM Judge: automatic personalized LLM judge design for evaluating natural language generation applicationscited by 7
  7. 2023Exploring Lottery Prompts for Pre-trained Language Modelscited by 5
  8. 2025Thunder-NUBench: A Benchmark for LLMs' Sentence-Level Negation Understandingcited by 5
  9. 2026Limited Linguistic Diversity in Embodied AI Datasetscited by 5
  10. 2023MAQA: A Multimodal QA Benchmark for Negationcited by 4
  11. 2025Do Methods to Jailbreak and Defend LLMs Generalize Across Languages?cited by 4
  12. 2024Balancing Exploration and Exploitation in LLM using Soft RLLF for Enhanced Negation Understandingcited by 3
  13. 2025Making Language Models Robust Against Negationcited by 3
  14. 2024Revisiting subword tokenization: A case study on affixal negation in large language modelscited by 2
  15. 2024SAGA: A Participant-specific Examination of Story Alternatives and Goal Applicability for a Deeper Understanding of Complex Eventscited by 2
  16. 2023A Framework For Refining Text Classification and Object Recognition from Academic Articlescited by 1
  17. 2023Snowman: A Million-scale Chinese Commonsense Knowledge Graph Distilled from Foundation Modelcited by 1
  18. 2023Detecting agreement in multi-party dialogue: evaluating speaker diarisation versus a procedural baseline to enhance user engagementcited by 1
  19. 2024Evaluating the Robustness of Adverse Drug Event Classification Models Using Templatescited by 1
  20. 2025Semantic Adapter for Universal Text Embeddings: Diagnosing and Mitigating Negation Blindness to Enhance Universalitycited by 1
  21. 2025Investigating Counterclaims in Causality Extraction from Textcited by 1
  22. 2026Negation Beyond the Verbal Channel: Temporal Multimodal Correlates in Dialoguecited by 1
  23. 2023Synthetic Dataset for Evaluating Complex Compositional Knowledge for Natural Language Inferencecited by 0
  24. 2024Generating Diverse Negations from Affirmative Sentencescited by 0
  25. 2025The Impact of Negated Text on Hallucination with Large Language Modelscited by 0
  26. 2026Interpreting Negation in GPT-2: Layer- and Head-Level Causal Analysiscited by 0
  27. 2026What CLIP Knows but Cannot Say: Recovering Negation from Frozen Intermediate Featurescited by 0