mlsanityData to 14 Oct 2026

Understanding by Understanding Not: Modeling Negation in Language Models

arXiv 2105.03519 ↗First posted 7 May 2021cs.CLcited by 73

Not on the map. Why?

Abstract (arXiv)

Negation is a core construction in natural language. Despite being very successful on many tasks, state-of-the-art pre-trained language models often handle negation incorrectly. To improve language models in this regard, we propose to augment the language modeling objective with an unlikelihood objective that is based on negated generic sentences from a raw text corpus. By training BERT with the resulting combined objective we reduce the mean top~1 error rate to 4% on the negated LAMA dataset. We also see some improvements on the negated NLI benchmarks.

Key facts. Cited by 73 papers in the map (data through 14 Oct 2026). Builds on 23 of its 43 references that are in the map. First posted on arXiv 7 May 2021.

Builds on

23 of 43 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  4. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  5. 2019Language Models as Knowledge Bases?cited by 1,936
  6. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  7. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  8. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 876
  9. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
  10. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 723
  11. 2019Neural Text Generation with Unlikelihood Trainingcited by 503
  12. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
11 more
  1. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 243
  2. 2018Targeted Syntactic Evaluation of Language Modelscited by 242
  3. 2018Colorless green recurrent networks dream hierarchicallycited by 213
  4. 2018Stress Test Evaluation for Natural Language Inferencecited by 213
  5. 2019oLMpics -- On what Language Model Pre-training Capturescited by 161
  6. 2019Good-Enough Compositional Data Augmentationcited by 158
  7. 2020Syntactic Data Augmentation Increases Robustness to Inference Heuristicscited by 86
  8. 2018Do Language Models Understand Anything? On the Ability of LSTMs to Understand Negative Polarity Itemscited by 36
  9. 2019A BERT Baseline for the Natural Questionscited by 33
  10. 2020An Analysis of the Utility of Explicit Negative Examples to Improve the Syntactic Abilities of Neural Language Modelscited by 7
  11. 2019Linguistic Analysis of Pretrained Sentence Encoders with Acceptability Judgmentscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 16; 2023: 24; 2024: 15; 2025: 9; 2026: 7.

73 papers.

Sort:

By area:

  1. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  2. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 449
  3. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 407
  4. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 302
  5. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 300
  6. 2022Language models show human-like content effects on reasoning taskscited by 173
  7. 2022ProsocialDialog: A Prosocial Backbone for Conversational Agentscited by 109
  8. 2022Chinese CLIP: Contrastive Vision-Language Pretraining in Chinesecited by 109
  9. 2024KMMLU: Measuring Massive Multitask Language Understanding in Koreancited by 91
  10. 2024ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabiccited by 88
  11. 2023Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Modelscited by 80
  12. 2023Language models are not naysayers: An analysis of language models on negation benchmarkscited by 74
38 more
  1. 2023Evaluating Superhuman Models with Consistency Checkscited by 47
  2. 2023Small Models are Valuable Plug-ins for Large Language Modelscited by 42
  3. 2022CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negationcited by 39
  4. 2022KOBEST: Korean Balanced Evaluation of Significant Taskscited by 37
  5. 2023Consistency Analysis of ChatGPTcited by 37
  6. 2023Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledgecited by 32
  7. 2023Joint Prompt Optimization of Stacked LLMs using Variational Inferencecited by 32
  8. 2023Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Modelscited by 32
  9. 2023Large Language Model Instruction Following: A Survey of Progresses and Challengescited by 31
  10. 2022E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoningcited by 28
  11. 2025Know "No" Better: A Data-Driven Approach for Enhancing Negation Awareness in CLIPcited by 28
  12. 2022Image Retrieval from Contextual Descriptionscited by 26
  13. 2023Learning Deductive Reasoning from Synthetic Corpus based on Formal Logiccited by 18
  14. 2023Limits for Learning with Language Modelscited by 17
  15. 2023This is not correct! Negation-aware Evaluation of Language Generation Systemscited by 17
  16. 2022Learn to Understand Negation in Video Retrievalcited by 16
  17. 2023Evaluating statistical language models as pragmatic reasonerscited by 15
  18. 2023Separating form and meaning: Using self-consistency to quantify task understanding across multiple sensescited by 15
  19. 2024Investigating and Addressing Hallucinations of LLMs in Tasks Involving Negationcited by 15
  20. 2025Potemkin Understanding in Large Language Modelscited by 15
  21. 2021Accurate, yet inconsistent? Consistency Analysis on Language Understanding Modelscited by 13
  22. 2023Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Modelscited by 12
  23. 2024TV-TREES: Multimodal Entailment Trees for Neuro-Symbolic Video Reasoningcited by 12
  24. 2022Beyond Distributional Hypothesis: Let Language Models Learn Meaning-Text Correspondencecited by 10
  25. 2023A negation detection assessment of GPTs: analysis with the xNot360 datasetcited by 10
  26. 2024Paraphrasing in Affirmative Terms Improves Negation Understandingcited by 8
  27. 2022Leveraging Affirmative Interpretations from Negation Improves Natural Language Understandingcited by 7
  28. 2024From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistencycited by 7
  29. 2022An Analysis of Social Biases Present in BERT Variants Across Multiple Languagescited by 6
  30. 2022Analyzing Semantic Faithfulness of Language Models via Input Intervention on Question Answeringcited by 6
  31. 2023Assessing Step-by-Step Reasoning against Lexical Negation: A Case Study on Syllogismcited by 6
  32. 2024Strong hallucinations from negation and how to fix themcited by 6
  33. 2026Beyond Calibration: Do a Typed-Decision Model's Probabilities Obey the Probability Axioms?cited by 6
  34. 2022Can Pretrained Language Models (Yet) Reason Deductively?cited by 5
  35. 2022Using Bottleneck Adapters to Identify Cancer in Clinical Notes under Low-Resource Constraintscited by 5
  36. 2023Exploring Lottery Prompts for Pre-trained Language Modelscited by 5
  37. 2025Thunder-NUBench: A Benchmark for LLMs' Sentence-Level Negation Understandingcited by 5
  38. 2023MAQA: A Multimodal QA Benchmark for Negationcited by 4

50 most cited of 73 All 73 (JSON)