# BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief

arXiv: https://arxiv.org/abs/2109.14723 · HTML: https://mlsanity.com/paper/2109.14723/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 29 Sep 2021.

## Builds on (most cited first, 21 of 21)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [Measuring and Improving Consistency in Pretrained Language Models](https://mlsanity.com/paper/2102.01017/) (2021)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [Transformers as Soft Reasoners over Language](https://mlsanity.com/paper/2002.05867/) (2020)
- [Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly](https://mlsanity.com/paper/1911.03343/) (2019)
- [Commonsense Knowledge Mining from Pretrained Models](https://mlsanity.com/paper/1909.00505/) (2019)
- [Unsupervised Commonsense Question Answering with Self-Talk](https://mlsanity.com/paper/2004.05483/) (2020)
- [Logic-Guided Data Augmentation and Regularization for Consistent Question Answering](https://mlsanity.com/paper/2004.10157/) (2020)
- [A Logic-Driven Framework for Consistency of Neural Models](https://mlsanity.com/paper/1909.00126/) (2019)
- [Tracking the World State with Recurrent Entity Networks](https://mlsanity.com/paper/1612.03969/) (2016)
- [Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data](https://mlsanity.com/paper/2009.09139/) (2020)
- [Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledge](https://mlsanity.com/paper/1808.08609/) (2018)
- [General-Purpose Question-Answering with Macaw](https://mlsanity.com/paper/2109.02593/) (2021)
- [Remembering for the Right Reasons: Explanations Reduce Catastrophic Forgetting](https://mlsanity.com/paper/2010.01528/) (2020)
- [Obtaining Faithful Interpretations from Compositional Neural Networks](https://mlsanity.com/paper/2005.00724/) (2020)

## Built on by (most cited first, 30 of 57)

- [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022)
- [Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts](https://mlsanity.com/paper/2305.13300/) (2023)
- [Evaluating the Ripple Effects of Knowledge Editing in Language Models](https://mlsanity.com/paper/2307.12976/) (2023)
- [Do Large Language Models Latently Perform Multi-Hop Reasoning?](https://mlsanity.com/paper/2402.16837/) (2024)
- [Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations](https://mlsanity.com/paper/2205.11822/) (2022)
- [A Review on Language Models as Knowledge Bases](https://mlsanity.com/paper/2204.06031/) (2022)
- [Towards Faithful Model Explanation in NLP: A Survey](https://mlsanity.com/paper/2209.11326/) (2022)
- [Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs](https://mlsanity.com/paper/2111.13654/) (2021)
- [Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions](https://mlsanity.com/paper/2207.14251/) (2022)
- [Empowering LLMs with Logical Reasoning: A Comprehensive Survey](https://mlsanity.com/paper/2502.15652/) (2025)
- [Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference](https://mlsanity.com/paper/2211.11875/) (2022)
- [Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning](https://mlsanity.com/paper/2210.12217/) (2022)
- [Towards Teachable Reasoning Systems: Using a Dynamic Memory of User Feedback for Continual System Improvement](https://mlsanity.com/paper/2204.13074/) (2022)
- [Two Failures of Self-Consistency in the Multi-Step Reasoning of LLMs](https://mlsanity.com/paper/2305.14279/) (2023)
- [Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback](https://mlsanity.com/paper/2112.09737/) (2021)
- [How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances](https://mlsanity.com/paper/2310.07343/) (2023)
- [Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge](https://mlsanity.com/paper/2305.05976/) (2023)
- [Belief Revision: The Adaptability of Large Language Models Reasoning](https://mlsanity.com/paper/2406.19764/) (2024)
- [Logically Consistent Language Models via Neuro-Symbolic Integration](https://mlsanity.com/paper/2409.13724/) (2024)
- [Factual Consistency of Multilingual Pretrained Language Models](https://mlsanity.com/paper/2203.11552/) (2022)
- [Separating form and meaning: Using self-consistency to quantify task understanding across multiple senses](https://mlsanity.com/paper/2305.11662/) (2023)
- [Language Models with Rationality](https://mlsanity.com/paper/2305.14250/) (2023)
- [Semantic Consistency for Assuring Reliability of Large Language Models](https://mlsanity.com/paper/2308.09138/) (2023)
- [Do language models have coherent mental models of everyday things?](https://mlsanity.com/paper/2212.10029/) (2022)
- [Improving Consistency in Large Language Models through Chain of Guidance](https://mlsanity.com/paper/2502.15924/) (2025)
- [Enabling LLM Knowledge Analysis via Extensive Materialization](https://mlsanity.com/paper/2411.04920/) (2024)
- [Investigating Context-Faithfulness in Large Language Models: The Roles of Memory Strength and Evidence Style](https://mlsanity.com/paper/2409.10955/) (2024)
- [GLUECons: A Generic Benchmark for Learning Under Constraints](https://mlsanity.com/paper/2302.10914/) (2023)
- [Desiderata for the Context Use of Question Answering Systems](https://mlsanity.com/paper/2401.18001/) (2024)
- [AI, Meet Human: Learning Paradigms for Hybrid Decision Making Systems](https://mlsanity.com/paper/2402.06287/) (2024)

## Abstract (arXiv)

Although pretrained language models (PTLMs) contain significant amounts of world knowledge, they can still produce inconsistent answers to questions when probed, even after specialized training. As a result, it can be hard to identify what the model actually "believes" about the world, making it susceptible to inconsistent behavior and simple errors. Our goal is to reduce these problems. Our approach is to embed a PTLM in a broader system that also includes an evolving, symbolic memory of beliefs -- a BeliefBank -- that records but then may modify the raw PTLM answers. We describe two mechanisms to improve belief consistency in the overall system. First, a reasoning component -- a weighted MaxSAT solver -- revises beliefs that significantly clash with others. Second, a feedback component issues future queries to the PTLM using known beliefs as context. We show that, in a controlled experimental setting, these two mechanisms result in more consistent beliefs in the overall system, improving both the accuracy and consistency of its answers over time. This is significant as it is a first step towards PTLM-based architectures with a systematic notion of belief, enabling them to construct a more coherent picture of the world, and improve over time without model retraining.