mlsanityData to 7 Oct 2026

Language Models As or For Knowledge Bases

arXiv 2110.04888 ↗First posted 10 Oct 2021cs.CL cs.AI cs.DBcited by 1

Not on the map. Why?

Abstract (arXiv)

Pre-trained language models (LMs) have recently gained attention for their potential as an alternative to (or proxy for) explicit knowledge bases (KBs). In this position paper, we examine this hypothesis, identify strengths and limitations of both LMs and KBs, and discuss the complementary nature of the two paradigms. In particular, we offer qualitative arguments that latent LMs are not suitable as a substitute for explicit KBs, but could play a major role for augmenting and curating KBs.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 23 of its 32 references that are in the map. First posted on arXiv 10 Oct 2021.

Builds on

23 of 32 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  5. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  6. 2019Language Models as Knowledge Bases?cited by 1,922
  7. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  8. 2018Object Hallucination in Image Captioningcited by 661
  9. 2021Editing Factual Knowledge in Language Modelscited by 548
  10. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
  11. 2020K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapterscited by 318
  12. 2017Recent Trends in Deep Learning Based Natural Language Processingcited by 262
11 more
  1. 2020Modifying Memories in Transformer Modelscited by 222
  2. 2020How Context Affects Language Models' Factual Predictionscited by 181
  3. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  4. 2020Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queriescited by 107
  5. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  6. 2020Entities as Experts: Sparse Memory Access with Entity Supervisioncited by 100
  7. 2019E-BERT: Efficient-Yet-Effective Entity Embeddings for BERTcited by 96
  8. 2020Language Models are Open Knowledge Graphscited by 95
  9. 2020Machine Knowledge: Creation and Curation of Comprehensive Knowledge Basescited by 55
  10. 2021A Study of the Quality of Wikidatacited by 35
  11. 2021Enriching a Model's Notion of Belief using a Persistent Memorycited by 4

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022Can Foundation Models Wrangle Your Data?cited by 173