mlsanityData to 14 Oct 2026

Can Large Language Models Truly Understand Prompts? A Case Study with Negated Prompts

arXiv 2209.12711 ↗First posted 26 Sep 2022cs.CLcited by 59

Not on the map. Why?

Abstract (arXiv)

Previous work has shown that there exists a scaling law between the size of Language Models (LMs) and their zero-shot performance on different downstream NLP tasks. In this work, we show that this phenomenon does not hold when evaluating large LMs on tasks with negated prompts, but instead shows an inverse scaling law. We evaluate 9 different tasks with negated prompts on (1) pretrained LMs (OPT & GPT-3) of varying sizes (125M - 175B), (2) LMs further pretrained to generalize to novel prompts (InstructGPT), (3) LMs provided with few-shot examples, and (4) LMs fine-tuned specifically on negated prompts; all LM types perform worse on negated prompts as they scale and show a huge performance gap between the human performance when comparing the average score on both original and negated prompts. By highlighting a critical limitation of existing LMs and methods, we urge the community to develop new approaches of developing LMs that actually follow the given instructions. We provide the code and the datasets to explore negated prompts at https://github.com/joeljang/negated-prompts-for-llms

Key facts. Cited by 59 papers in the map (data through 14 Oct 2026). Builds on 21 of its 26 references that are in the map. First posted on arXiv 26 Sep 2022.

Builds on

21 of 26 references.

References · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  10. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  11. 2022Emergent Abilities of Large Language Modelscited by 2,512
  12. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
9 more
  1. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  2. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  3. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  4. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 992
  5. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 417
  6. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 326
  7. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 320
  8. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 289
  9. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 25; 2024: 16; 2025: 5; 2026: 10.

59 papers.

Sort:

By area:

  1. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 302
  2. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 222
  3. 2024Programming Refusal with Conditional Activation Steeringcited by 206
  4. 2023Personality Traits in Large Language Modelscited by 169
  5. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  6. 2023Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Modelscited by 80
  7. 2023Language models are not naysayers: An analysis of language models on negation benchmarkscited by 74
  8. 2023Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewritingcited by 69
  9. 2023Active Learning Principles for In-Context Learning with Large Language Modelscited by 54
  10. 2023Controlled Text Generation via Language Model Arithmeticcited by 51
  11. 2023Do Models Really Learn to Follow Instructions? An Empirical Study of Instruction Tuningcited by 49
  12. 2023You don't need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instrumentscited by 48
38 more
  1. 2023Small Models are Valuable Plug-ins for Large Language Modelscited by 42
  2. 2024Social Bias Evaluation for Large Language Models Requires Prompt Variationscited by 40
  3. 2023Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledgecited by 32
  4. 2023Large Language Model Instruction Following: A Survey of Progresses and Challengescited by 31
  5. 2023Instruction-following Evaluation through Verbalizer Manipulationcited by 30
  6. 2024ALCM: Autonomous LLM-Augmented Causal Discovery Frameworkcited by 29
  7. 2023The Life Cycle of Knowledge in Big Language Models: A Surveycited by 23
  8. 2024Investigating and Addressing Hallucinations of LLMs in Tasks Involving Negationcited by 15
  9. 2023Are Language Models Worse than Humans at Following Prompts? It's Complicatedcited by 14
  10. 2022Despite "super-human" performance, current LLMs are unsuited for decisions about ethics and safetycited by 13
  11. 2023Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Modelscited by 12
  12. 2025Enhancing Vision-Language Compositional Understanding with Multimodal Synthetic Datacited by 12
  13. 2024CAT-LLM: Style-enhanced Large Language Models with Text Style Definition for Chinese Article-style Transfercited by 10
  14. 2024Using Large Language Models to Automate and Expedite Reinforcement Learning with Reward Machinecited by 9
  15. 2025Generative AI in Knowledge Work: Design Implications for Data Navigation and Decision-Makingcited by 9
  16. 2023Generalization Analogies: A Testbed for Generalizing AI Oversight to Hard-To-Measure Domainscited by 8
  17. 2024Say More with Less: Understanding Prompt Learning Behaviors through Gist Compressioncited by 8
  18. 2023Emergent inabilities? Inverse scaling over the course of pretrainingcited by 7
  19. 2023Prompts have evil twinscited by 7
  20. 2023Assessing Step-by-Step Reasoning against Lexical Negation: A Case Study on Syllogismcited by 6
  21. 2024RNR: Teaching Large Language Models to Follow Roles and Rulescited by 6
  22. 2026Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Trainingcited by 6
  23. 2023Probing Quantifier Comprehension in Large Language Models: Another Example of Inverse Scalingcited by 5
  24. 2024A Reality check of the benefits of LLM in businesscited by 5
  25. 2024On the Potential of Open-Vocabulary Models for Object Detection in Unusual Street Scenescited by 5
  26. 2024Smaller Large Language Models Can Do Moral Self-Correctioncited by 5
  27. 2025Rote Learning Considered Useful: Generalizing over Memorized Data in LLMscited by 5
  28. 2022Rarely a problem? Language models exhibit inverse scaling in their predictions following few-type quantifierscited by 4
  29. 2023Utilizing ChatGPT to Enhance Clinical Trial Enrollmentcited by 4
  30. 2023Can large language models generate salient negative statements?cited by 3
  31. 2024Can large language models understand uncommon meanings of common words?cited by 3
  32. 2024LLM Online Spatial-temporal Signal Reconstruction Under Noisecited by 3
  33. 2026Automated structural testing of LLM-based agents: methods, framework, and case studiescited by 3
  34. 2023Justifiable Artificial Intelligence: Engineering Large Language Models for Legal Applicationscited by 2
  35. 2024Pelican Soup Framework: A Theoretical Framework for Language Model Capabilitiescited by 2
  36. 2024Zero-shot prompt-based classification: topic labeling in times of foundation models in German Tweetscited by 2
  37. 2025Is In-Context Learning Learning?cited by 2
  38. 2025Follow My Lead: Logical Fallacy Classification with Knowledge-Augmented LLMscited by 1

50 most cited of 59 All 59 (JSON)