mlsanityData to 7 Oct 2026

Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs

arXiv 2111.13654 ↗First posted 26 Nov 2021cs.CL cs.AI cs.LGcited by 79

Not on the map. Why?

Abstract (arXiv)

Do language models have beliefs about the world? Dennett (1995) famously argues that even thermostats have beliefs, on the view that a belief is simply an informational state decoupled from any motivational state. In this paper, we discuss approaches to detecting when models have beliefs about the world, and we improve on methods for updating model beliefs to be more truthful, with a focus on methods based on learned optimizers or hypernetworks. Our main contributions include: (1) new metrics for evaluating belief-updating methods that focus on the logical consistency of beliefs, (2) a training objective for Sequential, Local, and Generalizing model updates (SLAG) that improves the performance of learned optimizers, and (3) the introduction of the belief graph, which is a new form of interface with language models that shows the interdependencies between model beliefs. Our experiments suggest that models possess belief-like qualities to only a limited extent, but update methods can both fix incorrect model beliefs and greatly improve their consistency. Although off-the-shelf optimizers are surprisingly strong belief-updating baselines, our learned optimizers can outperform them in more difficult settings than have been considered in past work. Code is available at https://github.com/peterbhase/SLAG-Belief-Updating

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). Builds on 26 of its 45 references that are in the map. First posted on arXiv 26 Nov 2021.

Builds on

26 of 45 references.

References · 26

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2019Language Models as Knowledge Bases?cited by 1,922
  4. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  5. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  6. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  7. 2019How Can We Know What Language Models Know?cited by 953
  8. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  9. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  10. 2019Designing and Interpreting Probes with Control Taskscited by 608
  11. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  12. 2021Editing Factual Knowledge in Language Modelscited by 548
14 more
  1. 2021Fast Model Editing at Scalecited by 491
  2. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 425
  3. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
  4. 2019KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representationcited by 356
  5. 2020K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapterscited by 318
  6. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  7. 2021Symbolic Knowledge Distillation: from General Language Models to Commonsense Modelscited by 270
  8. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 230
  9. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  10. 2020Modifying Memories in Transformer Modelscited by 222
  11. 2020Editable Neural Networkscited by 154
  12. 2020Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queriescited by 107
  13. 2021When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Datacited by 75
  14. 2021BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Beliefcited by 56

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 17; 2023: 22; 2024: 26; 2025: 9; 2026: 4.

79 papers.

Sort:

By area:

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  2. 2022Mass-Editing Memory in a Transformercited by 976
  3. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  4. 2022Memory-Based Model Editing at Scalecited by 443
  5. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 292
  6. 2022Post-hoc Concept Bottleneck Modelscited by 289
  7. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 267
  8. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  9. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  10. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 236
  11. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  12. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 211
38 more
  1. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  2. 2022A Review on Language Models as Knowledge Basescited by 156
  3. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 156
  4. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 140
  5. 2022Language Models as Agent Modelscited by 131
  6. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thoughtcited by 118
  7. 2022Meaning without reference in large language modelscited by 76
  8. 2024Are Large Language Models Consistent over Value-laden Questions?cited by 58
  9. 2023Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Modelscited by 46
  10. 2023GNNDelete: A General Strategy for Unlearning in Graph Neural Networkscited by 43
  11. 2023Meta-Learning Online Adaptation of Language Modelscited by 43
  12. 2024DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detectioncited by 41
  13. 2022Towards Teachable Reasoning Systems: Using a Dynamic Memory of User Feedback for Continual System Improvementcited by 37
  14. 2024Multi-hop Question Answering under Temporal Knowledge Editingcited by 36
  15. 2024DeepEdit: Knowledge Editing as Decoding with Constraintscited by 33
  16. 2024In-Context Editing: Learning Knowledge from Self-Induced Distributionscited by 33
  17. 2022Language Anisotropic Cross-Lingual Model Editingcited by 26
  18. 2024Propagation and Pitfalls: Reasoning-based Assessment of Knowledge Editing through Counterfactual Taskscited by 24
  19. 2024Fundamental Problems With Model Editing: How Should Rational Belief Revision Work in LLMs?cited by 24
  20. 2023The Life Cycle of Knowledge in Big Language Models: A Surveycited by 23
  21. 2024Long-form evaluation of model editingcited by 23
  22. 2022On the Domain Adaptation and Generalization of Pretrained Language Models: A Surveycited by 22
  23. 2023Memory Injections: Correcting Multi-Hop Reasoning Failures during Inference in Transformer-Based Language Modelscited by 21
  24. 2022BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Modelscited by 20
  25. 2022Unpacking Large Language Models with Conceptual Consistencycited by 18
  26. 2023Improved Logical Reasoning of Language Models via Differentiable Symbolic Programmingcited by 18
  27. 2023RECKONING: Reasoning through Dynamic Knowledge Encodingcited by 17
  28. 2023History Matters: Temporal Knowledge Editing in Large Language Modelcited by 17
  29. 2024Large Language Model Bias Mitigation from the Perspective of Knowledge Editingcited by 16
  30. 2023Language Models with Rationalitycited by 15
  31. 2024Moderator: Moderating Text-to-Image Diffusion Models through Fine-grained Context-based Policiescited by 15
  32. 2024Identifying and Mitigating Social Bias Knowledge in Language Modelscited by 13
  33. 2025CoME: An Unlearning-based Approach to Conflict-free Model Editingcited by 12
  34. 2024Leveraging Logical Rules in Knowledge Editing: A Cherry on the Topcited by 11
  35. 2023Information Association for Language Model Updating by Mitigating LM-Logical Discrepancycited by 10
  36. 2024Are language models rational? The case of coherence norms and belief revisioncited by 9
  37. 2024A Survey on Symbolic Knowledge Distillation of Large Language Modelscited by 9
  38. 2021Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledgecited by 8

50 most cited of 79 All 79 (JSON)