mlsanityData to 7 Oct 2026

Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge

arXiv 2305.05976 ↗First posted 10 May 2023cs.CLcited by 32

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have been widely studied for their ability to store and utilize positive knowledge. However, negative knowledge, such as "lions don't live in the ocean", is also ubiquitous in the world but rarely mentioned explicitly in the text. What do LLMs know about negative knowledge? This work examines the ability of LLMs to negative commonsense knowledge. We design a constrained keywords-to-sentence generation task (CG) and a Boolean question-answering task (QA) to probe LLMs. Our experiments reveal that LLMs frequently fail to generate valid sentences grounded in negative commonsense knowledge, yet they can correctly answer polar yes-or-no questions. We term this phenomenon the belief conflict of LLMs. Our further analysis shows that statistical shortcuts and negation reporting bias from language modeling pre-training cause this conflict.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 47 of its 84 references that are in the map. First posted on arXiv 10 May 2023.

Builds on

47 of 84 references.

References · 47

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  12. 2022Emergent Abilities of Large Language Modelscited by 2,483
35 more
  1. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  2. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  3. 2019Language Models as Knowledge Bases?cited by 1,922
  4. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  5. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  6. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  7. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  8. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  9. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  10. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  11. 2021MetaICL: Learning to Learn In Contextcited by 450
  12. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  13. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 309
  14. 2020Transformers as Soft Reasoners over Languagecited by 265
  15. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 240
  16. 2020How Context Affects Language Models' Factual Predictionscited by 181
  17. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  18. 2020A Survey of Knowledge-Enhanced Text Generationcited by 119
  19. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  20. 2020KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoningcited by 106
  21. 2021Understanding by Understanding Not: Modeling Negation in Language Modelscited by 73
  22. 2021BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Beliefcited by 56
  23. 2022Can Large Language Models Truly Understand Prompts? A Case Study with Negated Promptscited by 56
  24. 2021Why Machine Reading Comprehension Models Learn Shortcuts?cited by 40
  25. 2022An Analysis of Negation in Natural Language Understanding Corporacited by 39
  26. 2022Entailer: Answering Questions with Faithful and Truthful Chains of Reasoningcited by 38
  27. 2022CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negationcited by 38
  28. 2022E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoningcited by 28
  29. 2022Penguins Don't Fly: Reasoning about Generics through Instantiations and Exceptionscited by 17
  30. 2019Natural Language Generation Challenges for Explainable AIcited by 10
  31. 2020NegatER: Unsupervised Discovery of Negatives in Commonsense Knowledge Basescited by 9
  32. 2021Extending rational models of communication from beliefs to actionscited by 8
  33. 2022UnCommonSense: Informative Negative Knowledge about Everyday Conceptscited by 8
  34. 2022Breakpoint Transformers for Modeling and Tracking Intermediate Beliefscited by 6
  35. 2022Unsupervised Explanation Generation via Correct Instantiationscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 11; 2024: 13; 2025: 6; 2026: 2.

32 papers.

Sort:

By area:

  1. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  2. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  3. 2023Large Language Models are Temporal and Causal Reasoners for Video Question Answeringcited by 51
  4. 2024Benchmarking Knowledge Boundary for Large Language Models: A Different Perspective on Model Evaluationcited by 37
  5. 2023UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generationcited by 22
  6. 2024A Survey of Optimization-based Task and Motion Planning: From Classical To Learning Approachescited by 19
  7. 2023This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Modelscited by 18
  8. 2023In Search of the Long-Tail: Systematic Generation of Long-Tail Inferential Knowledge via Logical Rule Guided Searchcited by 9
  9. 2024AI, Meet Human: Learning Paradigms for Hybrid Decision Making Systemscited by 7
  10. 2024Leveraging Prompt-Based Large Language Models: Predicting Pandemic Health Decisions and Outcomes Through Social Media Languagecited by 6
  11. 2024Every Answer Matters: Evaluating Commonsense with Probabilistic Measurescited by 6
  12. 2023Statistical Knowledge Assessment for Large Language Modelscited by 5
20 more
  1. 2025How does Misinformation Affect Large Language Model Behaviors and Preferences?cited by 5
  2. 2025Thunder-NUBench: A Benchmark for LLMs' Sentence-Level Negation Understandingcited by 5
  3. 2025Generative AI for Self-Adaptive Systems: State of the Art and Research Roadmapcited by 5
  4. 2023KGQuiz: Evaluating the Generalization of Encoded Knowledge in Large Language Modelscited by 4
  5. 2023Gold: A Global and Local-aware Denoising Framework for Commonsense Knowledge Graph Noise Detectioncited by 4
  6. 2023Can large language models generate salient negative statements?cited by 3
  7. 2024Are LLMs Models of Distributional Semantics? A Case Study on Quantifierscited by 3
  8. 2024SAGA: A Participant-specific Examination of Story Alternatives and Goal Applicability for a Deeper Understanding of Complex Eventscited by 2
  9. 2023Snowman: A Million-scale Chinese Commonsense Knowledge Graph Distilled from Foundation Modelcited by 1
  10. 2023BOOST: Harnessing Black-Box Control to Boost Commonsense in LMs' Generationcited by 1
  11. 2024Rule or Story, Which is a Better Commonsense Expression for Talking with Large Language Models?cited by 1
  12. 2024ACCORD: Closing the Commonsense Measurability Gapcited by 1
  13. 2024Talking the Talk Does Not Entail Walking the Walk: On the Limits of Large Language Models in Lexical Entailment Recognitioncited by 1
  14. 2024Probing Causality Manipulation of Large Language Modelscited by 1
  15. 2025Estimating Commonsense Plausibility through Semantic Shiftscited by 1
  16. 2026Common Sense vs. Morality: The Curious Case of Narrative Focus Bias in LLMscited by 1
  17. 2024Carrot and Stick: Inducing Self-Motivation with Positive & Negative Feedbackcited by 0
  18. 2025CoVeR: Conformal Calibration for Versatile and Reliable Autoregressive Next-Token Predictioncited by 0
  19. 2025The Impact of Negated Text on Hallucination with Large Language Modelscited by 0
  20. 2026Tug-of-War within A Decade: Conflict Resolution in Vulnerability Analysis via Teacher-Guided Retrieval-Augmented Generationscited by 0