mlsanityData to 7 Oct 2026

Measuring and Improving Consistency in Pretrained Language Models

arXiv 2102.01017 ↗First posted 1 Feb 2021cs.CLcited by 361

Not on the map. Why?

Cited by 31 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Consistency of a model -- that is, the invariance of its behavior under meaning-preserving alternations in its input -- is a highly desirable property in natural language processing. In this paper we study the question: Are Pretrained Language Models (PLMs) consistent with respect to factual knowledge? To this end, we create ParaRel, a high-quality resource of cloze-style query English paraphrases. It contains a total of 328 paraphrases for 38 relations. Using ParaRel, we show that the consistency of all PLMs we experiment with is poor -- though with high variance between relations. Our analysis of the representational spaces of PLMs suggests that they have a poor structure and are currently not suitable for representing knowledge robustly. Finally, we propose a method for improving model consistency and experimentally demonstrate its effectiveness.

Key facts. Cited by 361 papers in the map (data through 7 Oct 2026). Builds on 49 of its 89 references that are in the map. First posted on arXiv 1 Feb 2021.

Builds on

49 of 89 references.

References · 49

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  9. 2015VQA: Visual Question Answeringcited by 2,489
  10. 2019Language Models as Knowledge Bases?cited by 1,922
  11. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  12. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
37 more
  1. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  2. 2019How Can We Know What Language Models Know?cited by 953
  3. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  4. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  5. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  6. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  7. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  8. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  9. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 350
  10. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  11. 2019Knowledge Enhanced Contextual Word Representationscited by 294
  12. 2018Analysis Methods in Neural Language Processing: A Surveycited by 276
  13. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  14. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 240
  15. 2018Targeted Syntactic Evaluation of Language Modelscited by 239
  16. 2019Assessing BERT's Syntactic Abilitiescited by 191
  17. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  18. 2020How Context Affects Language Models' Factual Predictionscited by 181
  19. 2017What do Neural Machine Translation Models Learn about Morphology?cited by 165
  20. 2019oLMpics -- On what Language Model Pre-training Capturescited by 161
  21. 2019Synthetic QA Corpora Generation with Roundtrip Consistencycited by 143
  22. 2020Injecting Numerical Reasoning Skills into Language Modelscited by 133
  23. 2020How Can We Accelerate Progress Towards Human-like Linguistic Generalization?cited by 114
  24. 2020Finding Universal Grammatical Relations in Multilingual BERTcited by 108
  25. 2019Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Modelcited by 102
  26. 2019E-BERT: Efficient-Yet-Effective Entity Embeddings for BERTcited by 96
  27. 2020Logic-Guided Data Augmentation and Regularization for Consistent Question Answeringcited by 77
  28. 2019Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanationscited by 71
  29. 2019A Logic-Driven Framework for Consistency of Neural Modelscited by 64
  30. 2019Do Neural Language Representations Learn Physical Commonsense?cited by 62
  31. 2019Cracking the Contextual Commonsense Code: Understanding Commonsense Reasoning Aptitude of Deep Contextual Representationscited by 24
  32. 2020Syntactic Search by Examplecited by 20
  33. 2016Integrating Distributional Lexical Contrast into Word Embeddings for Antonym-Synonym Distinctioncited by 13
  34. 2020Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually)cited by 12
  35. 2019Be Consistent! Improving Procedural Text Comprehension using Label Consistencycited by 10
  36. 2020Unsupervised Distillation of Syntactic Information from Contextualized Word Representationscited by 8
  37. 2020Neural Databasescited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 33; 2023: 91; 2024: 93; 2025: 76; 2026: 49.

361 papers.

Sort:

By area:

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  3. 2022Mass-Editing Memory in a Transformercited by 976
  4. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  5. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  6. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  7. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  8. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  9. 2021Unsolved Problems in ML Safetycited by 291
  10. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 284
  11. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  12. 2023Large Language Model Alignment: A Surveycited by 241
38 more
  1. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 211
  2. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 203
  3. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
  4. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  5. 2023Personality Traits in Large Language Modelscited by 166
  6. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 165
  7. 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 162
  8. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
  9. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  10. 2022A Review on Language Models as Knowledge Basescited by 156
  11. 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 155
  12. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  13. 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 123
  14. 2023The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reductioncited by 120
  15. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  16. 2023R-Tuning: Instructing Large Language Models to Say `I Don't Know'cited by 106
  17. 2024Looking Inward: Language Models Can Learn About Themselves by Introspectioncited by 105
  18. 2023Generating Benchmarks for Factuality Evaluation of Language Modelscited by 103
  19. 2024SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationalescited by 102
  20. 2022What Artificial Neural Networks Can Tell Us About Human Language Acquisitioncited by 97
  21. 2023The Generative AI Paradox: "What It Can Create, It May Not Understand"cited by 97
  22. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 93
  23. 2022Calibrating Factual Knowledge in Pretrained Language Modelscited by 92
  24. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  25. 2023SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistencycited by 89
  26. 2023Moral Foundations of Large Language Modelscited by 84
  27. 2021Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefscited by 79
  28. 2023Large Language Models and Knowledge Graphs: Opportunities and Challengescited by 71
  29. 2022Prompt Consistency for Zero-Shot Task Generalizationcited by 66
  30. 2022Understanding Transformer Memorization Recall Through Idiomscited by 62
  31. 2023Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmarkcited by 62
  32. 2024Factual Confidence of LLMs: on Reliability and Robustness of Current Estimatorscited by 59
  33. 2022Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictionscited by 57
  34. 2021BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Beliefcited by 56
  35. 2022Language Models in the Loop: Incorporating Prompting into Weak Supervisioncited by 51
  36. 2023Bias Against 93 Stigmatized Groups in Masked Language Models and Downstream Sentiment Classification Taskscited by 51
  37. 2023Multi-Agent Consensus Seeking via Large Language Modelscited by 50
  38. 2023Do Large Language Models Know about Facts?cited by 49

50 most cited of 361 All 361 (JSON)