mlsanityData to 7 Oct 2026

Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries

arXiv 2008.09036 ↗First posted 20 Aug 2020cs.CLcited by 107

Not on the map. Why?

Abstract (arXiv)

Pretrained language models have been suggested as a possible alternative or complement to structured knowledge bases. However, this emerging LM-as-KB paradigm has so far only been considered in a very limited setting, which only allows handling 21k entities whose single-token name is found in common LM vocabularies. Furthermore, the main benefit of this paradigm, namely querying the KB using a variety of natural language paraphrases, is underexplored so far. Here, we formulate two basic requirements for treating LMs as KBs: (i) the ability to store a large number facts involving a large number of entities and (ii) the ability to query stored facts. We explore three entity representations that allow LMs to represent millions of entities and present a detailed case study on paraphrased querying of world knowledge in LMs, thereby providing a proof-of-concept that language models can indeed serve as knowledge bases.

Key facts. Cited by 107 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 19; 2023: 28; 2024: 32; 2025: 13; 2026: 6.

107 papers.

Sort:

By area:

  1. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 238
  2. 2023PMET: Precise Model Editing in a Transformercited by 170
  3. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  4. 2022A Review on Language Models as Knowledge Basescited by 156
  5. 2021Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Modelscited by 132
  6. 2023Don't Trust ChatGPT when Your Question is not in English: A Study of Multilingual Abilities and Types of LLMscited by 103
  7. 2021Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?cited by 91
  8. 2023When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunitiescited by 89
  9. 2021BERTese: Learning to Speak to BERTcited by 82
  10. 2021Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefscited by 79
  11. 2023Large Language Models and Knowledge Graphs: Opportunities and Challengescited by 71
  12. 2023Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modelingcited by 56
38 more
  1. 2025How much do language models memorize?cited by 53
  2. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  3. 2023Do Large Language Models Know about Facts?cited by 49
  4. 2023Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Modelscited by 32
  5. 2021Relational World Knowledge Representation in Contextual Language Models: A Reviewcited by 31
  6. 2022SKILL: Structured Knowledge Infusion for Large Language Modelscited by 27
  7. 2024Learnable Privacy Neurons Localization in Language Modelscited by 26
  8. 2024Multimodal Prompt Learning with Missing Modalities for Sentiment Analysis and Emotion Recognitioncited by 26
  9. 2023A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepediacited by 24
  10. 2023The Life Cycle of Knowledge in Big Language Models: A Surveycited by 23
  11. 2022Plug-and-Play Adaptation for Continuously-updated QAcited by 21
  12. 2023Evaluating Language Models for Knowledge Base Completioncited by 20
  13. 2024Mixture of Parrots: Experts improve memorization more than reasoningcited by 20
  14. 2022LaKo: Knowledge-driven Visual Question Answering via Late Knowledge-to-Text Injectioncited by 18
  15. 2024Can Language Models Act as Knowledge Bases at Scale?cited by 17
  16. 2024The Hallucinations Leaderboard -- An Open Effort to Measure Hallucinations in Large Language Modelscited by 17
  17. 2022Factual Consistency of Multilingual Pretrained Language Modelscited by 15
  18. 2024Monotonic Representation of Numeric Properties in Language Modelscited by 15
  19. 2024MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Modelscited by 15
  20. 2021Interpreting Language Models Through Knowledge Graph Extractioncited by 14
  21. 2023TempEL: Linking Dynamically Evolving and Newly Emerging Entitiescited by 14
  22. 2024Tracing the Roots of Facts in Multilingual Language Models: Independent, Shared, and Transferred Knowledgecited by 14
  23. 2023Rethinking Language Models as Symbolic Knowledge Graphscited by 12
  24. 2024Human-inspired Perspectives: A Survey on AI Long-term Memorycited by 12
  25. 2021Static Embeddings as Efficient Knowledge Bases?cited by 11
  26. 2023SpEL: Structured Prediction for Entity Linkingcited by 11
  27. 2024Scope Ambiguities in Large Language Modelscited by 11
  28. 2024Probing Language Models on Their Knowledge Sourcecited by 11
  29. 2025Do Large Language Models Know How Much They Know?cited by 11
  30. 2024When Context Leads but Parametric Memory Follows in Large Language Modelscited by 10
  31. 2024The Factuality of Large Language Models in the Legal Domaincited by 10
  32. 2024The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspacescited by 10
  33. 2023Evaluating the Knowledge Base Completion Potential of GPTcited by 9
  34. 2024Evaluating Robustness of Generative Search Engine on Adversarial Factual Questionscited by 9
  35. 2024Failing Forward: Improving Generative Error Correction for ASR with Synthetic Data and Retrieval Augmentationcited by 9
  36. 2025RealFactBench: A Benchmark for Evaluating Large Language Models in Real-World Fact-Checkingcited by 9
  37. 2021An Empirical Study on Few-shot Knowledge Probing for Pretrained Language Modelscited by 8
  38. 2022Revision Transformers: Instructing Language Models to Change their Valuescited by 8

50 most cited of 107 All 107 (JSON)