# Completeness, Recall, and Negation in Open-World Knowledge Bases: A Survey

arXiv: https://arxiv.org/abs/2305.05403 · HTML: https://mlsanity.com/paper/2305.05403/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 138 references that are in the map.
- First posted on arXiv 9 May 2023.

## Builds on (most cited first, 30 of 37)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://mlsanity.com/paper/2012.00955/) (2020)
- [K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters](https://mlsanity.com/paper/2002.01808/) (2020)
- [Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models](https://mlsanity.com/paper/2005.00683/) (2020)
- [UniK-QA: Unified Representations of Structured and Unstructured Knowledge for Open-Domain Question Answering](https://mlsanity.com/paper/2012.14610/) (2020)
- [Machine Knowledge: Creation and Curation of Comprehensive Knowledge Bases](https://mlsanity.com/paper/2009.11564/) (2020)
- [GenIE: Generative Information Extraction](https://mlsanity.com/paper/2112.08340/) (2021)
- [Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction](https://mlsanity.com/paper/1906.08646/) (2019)
- [Prompting as Probing: Using Language Models for Knowledge Base Construction](https://mlsanity.com/paper/2208.11057/) (2022)
- [Commonsense Properties from Query Logs and Question Answering Forums](https://mlsanity.com/paper/1905.10989/) (2019)
- [Conversational Question Answering on Heterogeneous Sources](https://mlsanity.com/paper/2204.11677/) (2022)
- [Advanced Semantics for Commonsense Knowledge Extraction](https://mlsanity.com/paper/2011.00905/) (2020)
- [OPIEC: An Open Information Extraction Corpus](https://mlsanity.com/paper/1904.12324/) (2019)
- [Evaluating Language Models for Knowledge Base Completion](https://mlsanity.com/paper/2303.11082/) (2023)
- [Refined Commonsense Knowledge from Large-Scale Web Contents](https://mlsanity.com/paper/2112.04596/) (2021)
- [UNIQORN: Unified Question Answering over RDF Knowledge Graphs and Natural Language Text](https://mlsanity.com/paper/2108.08614/) (2021)
- [Predicting Completeness in Knowledge Bases](https://mlsanity.com/paper/1612.05786/) (2016)
- [From Data Fusion to Knowledge Fusion](https://mlsanity.com/paper/1503.00302/) (2015)
- [Negative Statements Considered Useful](https://mlsanity.com/paper/2001.04425/) (2020)
- [NegatER: Unsupervised Discovery of Negatives in Commonsense Knowledge Bases](https://mlsanity.com/paper/2011.07497/) (2020)
- [UnCommonSense: Informative Negative Knowledge about Everyday Concepts](https://mlsanity.com/paper/2208.09292/) (2022)
- [Non-Parametric Class Completeness Estimators for Collaborative Knowledge Graphs -- The Case of Wikidata](https://mlsanity.com/paper/1909.01109/) (2019)

## Built on by (most cited first, 7 of 7)

- [Enabling LLM Knowledge Analysis via Extensive Materialization](https://mlsanity.com/paper/2411.04920/) (2024)
- [Can large language models generate salient negative statements?](https://mlsanity.com/paper/2305.16755/) (2023)
- [Large Language Models for Knowledge Graph Embedding: A Survey](https://mlsanity.com/paper/2501.07766/) (2025)
- [EMERGE: A Benchmark for Updating Knowledge Graphs with Emerging Textual Knowledge](https://mlsanity.com/paper/2507.03617/) (2025)
- [gHAWK: Local and Global Structure Encoding for Scalable Training of Graph Neural Networks on Knowledge Graphs](https://mlsanity.com/paper/2512.08274/) (2025)
- [MMCOMET: A Large-Scale Multimodal Commonsense Knowledge Graph for Contextual Reasoning](https://mlsanity.com/paper/2603.01055/) (2026)
- [Three Types of Negation of Triple and its Elements and an Extension of Triple](https://mlsanity.com/paper/2609.08271/) (2026)

## Abstract (arXiv)

General-purpose knowledge bases (KBs) are a cornerstone of knowledge-centric AI. Many of them are constructed pragmatically from Web sources, and are thus far from complete. This poses challenges for the consumption as well as the curation of their content. While several surveys target the problem of completing incomplete KBs, the first problem is arguably to know whether and where the KB is incomplete in the first place, and to which degree. In this survey we discuss how knowledge about completeness, recall, and negation in KBs can be expressed, extracted, and inferred. We cover (i) the logical foundations of knowledge representation and querying under partial closed-world semantics; (ii) the estimation of this information via statistical patterns; (iii) the extraction of information about recall from KBs and text; (iv) the identification of interesting negative statements; and (v) relaxed notions of relative recall. This survey is targeted at two types of audiences: (1) practitioners who are interested in tracking KB quality, focusing extraction efforts, and building quality-aware downstream applications; and (2) data management, knowledge base and semantic web researchers who wish to understand the state of the art of knowledge bases beyond the open-world assumption. Consequently, our survey presents both fundamental methodologies and their working, and gives practice-oriented recommendations on how to choose between different approaches for a problem at hand.